Response

W1

　　Thank you for your valuable suggestions. We have optimised the manuscript structure as follows:

1. We have relocated several key experiments from the appendix to the main experimental section.

Considering that certain key experiments, such as ablation studies and hyperparameter analyses, are crucial for validating the model's effectiveness, we have moved these specific experiments from the appendix to the main experimental section. We have also supplemented them with more detailed design descriptions and result analyses to improve the completeness and readability of the main text.

2. We have added an "Experimental Overview" subsection.

To improve the overall readability of the main text, we add a new subsection titled "Experimental Overview" to briefly summarize the design and key findings of each experiment. At the same time, we provide clear guidance directing readers to the more detailed figures and analyses in the appendix, enabling a more efficient understanding of the core contributions.

Manuscript revision: We have added subsections "4.4 Experimental Overview", "4.5 Ablation Experiments", and "4.6 Hyperparameter Effects" in the revised manuscript.

---

W2 & Q1

We have streamlined the description of two-level entropy-based sample filtering strategy and added a schematic diagram to aid intuitive understanding. This mechanism is designed to prioritize the selection of samples with high unimodal entropy and low multimodal entropy for fine-tuning. Specifically, it includes the following two aspects:

I. Unimodal entropy $Ent(x_i)$ measures a sample's reliance on a single modality. Higher values indicate suitability for fine‐tuning.

• Low $Ent(x_i)$: Model can predict using one dominant modality, implying limited multimodal information.
• High $Ent(x_i)$: Model likely integrates multiple modalities for prediction, indicating rich multimodal information.

II. Multimodal entropy $Ent(x)$ reflects prediction certainty. Lower values indicate suitability for fine‐tuning.

• Low $Ent(x)$: Predictions are more certain. This helps reduce uncertainty in target domain predictions.
• High $Ent(x)$: Predictions are uncertain. The uncertainty in predictions may introduce misleading information and compromise model stability, therefore making such samples unsuitable for fine-tuning.

Manuscript revision: We have updated "3.3.1 Two-level Entropy-based Sample Filtering" and added the schematic diagram.

---

W3

　　Thank you for the detailed feedback on Figure 1's colour scheme. We have simplified the colour palette and increased foreground-background contrast to more clearly delineate components and overall structure.

---

Q2

We determine the number of iterations $iter$ experimentally.

In the experiment labeled "The value of the iter," shown in the lower-left panel of Figure 2, we systematically evaluate performance under various settings and ultimately set $iter=7$. Using this selected value, Figure 6 then illustrates how model performance evolves from iteration 1 through 7, showing steady improvement and peak performance at iteration 7.

---

Q3

　　Thank you for your question. Below is our response:

1. The entropy threshold is determined jointly by batch‐level statistics and hyperparameter tuning.

Inspired by Chebyshev's inequality, we set the threshold to the batch's mean entropy plus $n$ times its variance, where $n$ is a hyperparameter controlling the strictness of sample filtering. This design aims to retain high‑quality samples while suppressing interference from outlier samples. The detailed methodology for selecting the entropy threshold is provided in "A.3 Selection of Entropy Threshold", and we will emphasize this discussion more prominently in the main text to clarify its role within the overall method.

2. Both overly strict and overly loose thresholds degrade performance, and thus an appropriate threshold needs to be determined through empirical evaluation.

• Too strict: Too few samples pass the filter, limiting learning of the target‐domain distribution.
• Too loose: A large number of samples are accepted, including outliers, which increases the risk of adapting to noisy data.

Response

W1

　　Thank you for your valuable suggestions. We have optimized the manuscript structure as follows:

1. We have relocated several key experiments from the appendix to the main experimental section.

   Considering that certain key experiments, such as ablation studies and hyperparameter analyses, are crucial for validating the model's effectiveness, we have moved these specific experiments from the appendix to the main experimental section. We have also supplemented them with more detailed design descriptions and result analyses to improve the completeness and readability of the main text.

2. We have added an "Experimental Overview" subsection.

   To improve the overall readability of the main text, we add a new subsection titled "Experimental Overview" to briefly summarize the design and key findings of each experiment. At the same time, we provide clear guidance directing readers to the more detailed figures and analyses in the appendix, enabling a more efficient understanding of the core contributions.

Manuscript revision: We have added subsections "4.4 Experimental Overview", "4.5 Ablation Experiments", and "4.6 Hyperparameters Effection" in the revised manuscript.

---

W2

　　Thank you for your valuable feedback regarding the basis for the selection of some hyperparameters and the summary of experiments. We have added relevant content in the main text, as outlined below:

1. Explanation of hyperparameter selection.

   In "4.6 Hyperparameter Effects" of the main text, We have supplemented the basis for choosing key hyperparameters, such as the entropy threshold and smoothing factor, and analyzed their performance curves under varying values. The basis for selecting the entropy threshold and the smoothing factor can be found in our responses to Q3 and Q4.

2. Summary and connection of hyperparameter experiments.

   We have added a new subsection "4.4 Experimental Overview," which briefly summarizes the impact of each hyperparameter on model performance and directs readers to the corresponding experimental details in the manuscript.

Manuscript revision: We have added subsections "4.4 Experimental Overview", "4.5 Ablation Experiments", and "4.6 Hyperparameters Effection" in the revised manuscript.

---

Q1

The case where a modality is completely missing rarely occurs in physiological signal acquisition experiments.

In actual physiological signal collection, complete modality missing is usually detected in time and fixed by inspecting or replacing the device. Therefore, our study mainly focuses on the more common problem of partial modality data missing.

---

Q2

We regulate the contribution of different modalities in the model by adjusting the loss weight of the auxiliary task and the hyperparameter of Mutual Information Sharing Across Modalities.

These parameters reflect the relative importance and influence of each modality. To further enhance interpretability, we plan to add a weight-visualization analysis for the inter-modal BiMamba in a future version, which will intuitively illustrate each modality's specific contribution.

---

Q3

The entropy threshold is determined jointly by batch-level statistics and hyperparameter tuning.

Inspired by Chebyshev's inequality, we set the threshold to the batch's mean entropy plus $n$ times its variance, where $n$ is a hyperparameter controlling the strictness of sample filtering. This design aims to retain high‑quality samples while suppressing interference from outlier samples. The detailed methodology for selecting the entropy threshold is provided in "A.3 Selection of Entropy Threshold", and we will emphasize this discussion more prominently in the main text to clarify its role within the overall method.

---

Q4

The initial value $\beta$ and the smoothing factor $\beta_t$ influence both the number and quality of samples entering the adaptation process at each iteration. The smoothing factor $\beta_t$ is jointly determined by the initial value $\beta$ and the growth strategy, with the specific effects described as follows:

I. Initial $\beta$: A smaller $\beta$ yields a stricter initial threshold, retaining only samples with rich multimodal information and high prediction confidence, thereby enhancing stability at early testing phase. A larger $\beta$ relaxes the threshold, potentially admitting low‐quality samples too soon and disrupting learned representations, which may degrade performance.

The optimal value of $\beta$ is determined experimentally and set to 0.2, with relevant results shown at "The value of the $\beta$" in the lower right corner of Figure 2.

II. Growth strategy: After determining a suitable $\beta$, we adopt the following growth strategy: $\beta_t = \beta + (1 - \beta)\cdot\frac{t}{\mathrm{iter}}$ where $\beta_t$ increases linearly with iterations, gradually relaxing the entropy threshold. This process allows more samples to be sequentially incorporated into the adaptation process and enables the model to more effectively cover the target domain.

In summary, by setting an appropriate initial $\beta$ and growth rate, this strategy balances early robustness with later adaptability, facilitating a smooth, high-quality model adaptation.

---

Q5

In our paper, we emphasize both Mamba’s strength in modeling long‑range dependencies and its ability to capture inter‑modal correlations.

Specifically, besides effectively capturing long-range temporal dependencies, its explicit global state variables, as part of the state space model (SSM), also effectively characterize correlations between different modalities. Based on these considerations, we ultimately select Mamba as the core component of the backbone network. As briefly mentioned in line 82 of the introduction, this topic is initially discussed. In subsequent versions, we will further elaborate to aid readers' deeper understanding.

Response

W1

　　Thank you very much for your attention to and detailed review of our conceptual novelty. In response to your questions, we are here to further clarify and supplement two aspects to more clearly demonstrate our innovations in the two-level entropy-based sample filtering strategy and its synergy with the overall architecture:

1. Our work represents an extension and expansion of TTA strategy within the field.

   Although entropy-based selective adaptation strategies have been extensively validated in computer vision and related domains, the specific downstream task of “multimodal physiological signals + emotion recognition + data missing” remains largely unexplored. Compared to existing EEG‐based TTA methods such as ATTA[1] and OTTA[2], our method uses entropy in a more distinctive and targeted manner. ATTA updates the adaptive model via unimodal cross-entropy, and OTTA minimizes only classification entropy. In contrast, we introduce both unimodal entropy and multimodal entropy, where the former captures the richness of emotional information present in each sample’s multimodal physiological signals and the latter gauges the model’s confidence in predicting the emotional state. Based on these, we construct a soft-threshold filtering strategy that dynamically removes samples whose emotional features degrade due to missing key information, more finely distinguishing sample quality across emotions. This ensures stability during adaptation while more effectively improving cross‐domain emotion recognition accuracy.

2. The filtering mechanism does not function in isolation. Instead, it serves as an integral part of the overall architecture,working in concert with other modules to jointly enhance the model's adaptive capability during the testing phase.

   I. Synergistic Effect between Two-level Entropy-based Sample Filtering and Mutual Information Sharing Across Modalities. Our multimodal TTA method comprises two complementary components: a global two-level entropy-based sample filtering strategy and a local cross modalities strategy.

   • The global strategy uses a soft threshold jointly constructed from unimodal and multimodal entropy to select a subset of information‐rich, high‐confidence samples, thereby ensuring data quality during adaptation.
   • The local strategy, based on these reliable samples, further promotes cross‐modal alignment by minimizing the KL divergence among prediction distributions of each modality.
     Together, they achieve progressive optimization from global sample filtering to local modal alignment.

   II. Synergistic Effect between the Multimodal BiMamba Backbone and TTA Method with Two-level Entropy-based Sample Filtering and Mutual Information Sharing Across Modalities.

   The multimodal BiMamba backbone network first captures emotional features along temporal and channel dimensions, mining long-range temporal dependencies and cross-modal correlations to generate strong base representations.Building on these high-quality features, the TTA method applies the two-level entropy-based sample filtering and mutual information sharing across modalities: the former ensures adaptation only on reliable samples, and the latter dynamically adjusts inter-modal information flow within each sample, mitigating interference from imbalanced modality information. The network and TTA collaborate across feature construction and test‐time adaptation to enhance emotion recognition performance.

---

W2 & Q1

　　As no existing work has explored Mamba-Transformer hybrids in physiological-signal emotion recognition, this direction remains unexplored and was not included in our original comparisons. Following your suggestion, we have added three recent SSM-Transformer approaches for time-series modelling as supplementary baselines, including MambaFormer and SST as Mamba-Transformer hybrid models and Heracles as a Transformer-based SSM method. And we have incorporated the results and analysis of these baselines into "4.3 Experiment Analysis." Thank you for your valuable suggestion.

Table: Comparative analysis of accuracy (Valence, Arousal) for different emotion baselines on DEAP and MAHNOB-HCI datasets. * indicates p<0.05.

Experimental analysis: MambaFormer embeds the Mamba module into each Transformer encoder’s feed‑forward layer to create a unified long‑and‑short‑range architecture. SST introduces a Mamba‑based global pattern expert alongside a windowed Transformer local variation expert and dynamically fuses their outputs via a long‑short‑range router. Heracles leverages a real‑valued Hartley‑kernel SSM to capture global sequence dependencies, a localized convolutional network to extract fine‑grained details and attention mechanisms in its deeper layers to enable rich temporal token interactions. These models exhibit strong long‑short‑range modeling capabilities. However, their designs focus solely on temporal dynamics and do not explicitly model cross‑channel interactions, which prevents them from capturing inter‑modal relationships and limits their effectiveness in multimodal emotion recognition tasks. Moreover, the complexity of these hybrid architectures can introduce a greater risk of overfitting on relatively small physiological‑signal datasets, further affecting their performance.

Manuscript revision: We have updated the results in Table 1 and revised Section 4.3 "Experiment Analysis" accordingly.

---

---

References

[1] Jia Z, Yang X, Zhou C, et al. Atta: Adaptive Test-Time Adaptation for Multi-Modal Sleep Stage Classification. Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI), 2024.

[2] Guo H, Li C, Peng H, Han Z, Chen X. Test Time Adaptation for Cross-Domain Sleep Stage Classification. Biomedical Signal Processing and Control, 2025.

[3] Xu X, Liang Y, Huang B, Lan Z, Shu K. Integrating Mamba and Transformer for Long-Short Range Time Series Forecasting. CoRR, 2024.

[4] Badri N. Patro, Suhas Ranganath, Vinay P. Namboodiri, Vijay S. Agneeswaran. Heracles: A Hybrid SSM-Transformer Model for High-Resolution Image and Time-Series Analysis. arXiv, 2024.

Response

W1&Q1:

　　Thank you for your suggestions on the methodology section of our paper. To clarify the design thoughts of the BiMamba module, we have added the corresponding explanation in "3.2.1 Intra-modal BiMamba Module" and "3.2.2 Inter-modal BiMamba Module" of the main text. The module's workflow comprises the following three steps: (1) Gating mechanism $\to$ (2) Bidirectional state-space modelling $\to$ (3) Linear projection and residual connection. Specifically, the intuition behind them is:

1. Gating mechanism: This mechanism assigns learnable weights to each channel or time step, adaptively emphasizing emotion-relevant representations while suppressing irrelevant information.

   In particular, physiological signals not only contain key features such as abrupt emotional change points and dynamic regions that faithfully reflect emotional fluctuations, but also often include environmental noise and repetitive resting state potentials that constitute redundant, task-irrelevant information [1][2]. The key features provide discriminative cues for emotion recognition, while the redundant information may obscure or interfere with the model's perception of sudden emotional changes. Therefore, the gating mechanism introduces learnable weight coefficients during feature extraction to dynamically adjust the response strength of each channel and time step, allocating higher weights at emotional change points to amplify discriminative signals and allocating lower weights in resting state or noisy segments to suppress interference. In this way, the model focuses on the most emotionally informative dynamic features, thereby improving recognition accuracy.

2. Bidirectional state-space modelling: Forward and backward features are constructed in parallel to capture richer contextual relationships.

   I. Bidirectional State-Space Modeling Along the Temporal Dimension: Emotion recognition is inherently a sequential modeling task, since emotional states emerge from temporally dependent dynamics rather than isolated points[3]. Unidirectional modeling often struggles to capture strong temporal correlations, such as lagged or precursory signals, leading to overlooked emotional cues[4]. To address this, BiMamba first applies linear mapping, convolution, and activation for initial local feature extraction, then deploys forward and backward state-space model SSM branches in parallel along the time dimension. This bidirectional modeling perceives complete contextual dependencies from both temporal paths, enhancing the model's understanding and discrimination of cross-time emotional dynamics.

   II. Bidirectional Cross-Modal Interaction Along the Channel Dimension: Emotional states commonly exhibit correlations across multiple physiological modalities, which are reflected at the channel level. For instance, increases in EEG $\theta$- and $\delta$-band power often exhibit positive Pearson correlations with galvanic skin response (GSR) conductance peaks and negative correlations with electrocardiogram (ECG) heart-rate variability during emotional arousal[5]. In the forward branch, the representation of the $i$-th modality references the hidden states of preceding modalities from 1 through $i-1$. In the backward branch, it is supplemented by the hidden states of subsequent modalities from $M$ down to $i+1$, where $M$ is the total number of modalities. This bidirectional information flow enables each modality to leverage context from earlier modalities and complete its features using later ones, generating complementary expressions for the same modality and strengthening inter-modal associations and fusion.

3. Linear projection and residual connections: Integrate bidirectional features and alleviate the vanishing gradient problem.

   We first fuse the forward feature $h_i^\rightarrow$ and backward feature $h_i^\leftarrow$ by averaging, then apply a linear mapping to merge features and perform dimensionality transformation. Finally, a residual connection is introduced to alleviate gradient vanishing.

Manuscript revision: We have incorporated the above design thoughts into "3.2.1 Intra-modal BiMamba Module" and "3.2.2 Inter-modal BiMamba Module" to aid reader comprehension.

---

---

References

[1] Barradas, I. et al. Dynamic emotion intensity estimation from physiological signals. PLOS ONE, 2025.

[2] Brake, N., Duc, F., Rokos, A., Arseneau, F., Shahiri, S., Khadra, A., & Plourde, G., A neurophysiological basis for aperiodic EEG and the background spectral trend, Nature Communications, 2024.

[3] Hofmann, S.M., Klotzsche, F., Mariola, A., Nikulin, V., Villringer, A., & Gaebler, M.. Decoding subjective emotional arousal from EEG during an immersive virtual reality experience. eLife, 2021.

[4] Zhang X, Zhang Q, Liu H, et al. Mamba in speech: Towards an alternative to self-attention. IEEE Transactions on Audio, Speech and Language Processing, 2025.

[5] Miranda-Correa JA, Abadi MK, Sebe N, Patras I. AMIGOS: A dataset for affect, personality and mood research on individuals and groups. IEEE Transactions on Affective Computing, 2018.