Joint Modeling of fMRI and EEG Imaging Using Ordinary Differential Equation-Based Hypergraph Neural Networks

Comment: We sincerely appreciate your meticulous review of our manuscript. Your valuable suggestions have helped a lot in improving the Clarity and Rigor of our work.

Q1: The paper lacks a dedicated related work section. Incorporating a more thorough review of existing studies on fMRI-EEG fusion, especially on asynchronous modeling, would strengthen the background and better contextualize the novelty of this work.

A1: We thank the reviewer for this essential suggestion. We acknowledge the need to rigorously position FE-NET within the field and will add a dedicated subsection (1.2: Related Work). Below is our revision strategy to address this gap:

1. New Section 1.2: "Related Work on Multimodal Fusion and Asynchronous Modeling"

Structure and key additions:

1. Synchronous fMRI-EEG Fusion:

   "Early fusion methods required strict temporal synchronization. Advanced deep learning approaches improved synchronization through attention mechanisms but remained limited to temporally aligned data. These cannot handle real-world asynchronous acquisitions."

2. Asynchronous Modeling Attempts:

   "Some studies used kernel methods to align asynchronous data post-hoc but lost dynamic interactions. Others employed self-supervised techniques but treated modalities independently without joint topological modeling. Prior work has not resolved asynchronous fusion at the graph-structure level."

3. Neural ODEs in Neuroimaging:

   "Neural ODEs have modeled single-modality dynamics in fMRI data. Spatio-temporal extensions capture dynamic patterns but ignore cross-modal asynchrony. None have adapted ODEs for multimodal hypergraph evolution."

4. Hypergraph Neuroimaging:

   "Existing hypergraph neural networks modeled fMRI data but omitted EEG integration. Other works fused MRI modalities via hypergraphs but required temporal alignment. FE-NET is the first to leverage hypergraphs for asynchronous fMRI-EEG integration."

---

2. Addressing the Asynchronous Modeling Gap

• Temporal Flexibility:

  "Unlike methods that process modalities separately, our Neural ODE-based FED module continuously aligns fast EEG and slow fMRI dynamics within a unified latent space."

• Structural Innovation:

  "While existing approaches aligned async data through static methods, FE-NET’s hypergraphs encode many-to-many cross-modal interactions, preserving complex neurodynamics."

---

3. Revisions Commitment

In the manuscript:

• Add ~400 words in Section 1.2 (preceding the "Our Contributions" paragraph)
• Include key references covering:
  • Synchronous and asynchronous fusion techniques
  • ODE applications in neuroimaging
  • Hypergraph-based neuroimaging studies
• Novelty statement:

  "To our knowledge, this is the first framework to simultaneously resolve: (a) Asynchronous fMRI-EEG fusion via hypergraph-structured Neural ODEs, (b) GAN-based generation of multimodal hypergraphs, and (c) Dynamic embedding of cross-modal interactions without temporal resampling."

Summary: This revision will:

1. Formally establish FE-NET’s pioneering role in asynchronous fMRI-EEG hypergraph fusion
2. Provide scholarly grounding for technical innovations
3. Explicitly contrast with prior approaches to highlight topological and temporal advantages

We affirm the revised manuscript will comprehensively contextualize our contributions within the field.

Q2: The motivation for using GANs in the hypergraph generation module is not clearly explained. It would be helpful if the authors could elaborate on why a GAN-based approach is necessary or beneficial in this setting, compared to more standard alternatives.

A2: We appreciate this critical inquiry into our design choice. The GAN-based approach in the FEH module addresses three fundamental challenges in fMRI-EEG hypergraph generation that deterministic or non-adversarial methods cannot resolve. Below, we clarify the rationale with empirical and theoretical evidence:

---

1. Key Challenges Necessitating GANs

---

2. Unique Advantages of GANs in FEH

• OFEI-IHEN Synchronization:
  The GAN framework unifies our two innovations:
  1. OFEI ensures structural consistency (optimal hypergraph isomorphism).
  2. IHEN learns dynamic node-hyperedge interactions (Eq. 5).
  • The discriminator’s random walk loss jointly supervises OFEI and IHEN, forcing them to collaborate on biologically plausible hypergraphs.
• Mode Collapse Prevention:
  fMRI-EEG datasets exhibit high inter-subject variability. Our path-based discriminator (evaluating whole-graph connectivity) avoids trivial solutions by:
  • Comparing global topology, not just local edges.
  • Enforcing diversity through stochastic walks.

---

3. Neurobiological Motivation

The discriminator’s FC-referenced realism check mirrors how neuroscientists validate connectivity:

"Just as researchers compare generated networks against gold-standard fMRI FC, our discriminator evaluates whether synthetic hypergraphs exhibit the same macroscale organizational principles "

This adversarial "peer review" ensures hypergraphs reflect real brain network properties, not just data-fitting artifacts.

---

5. Revisions Commitment

We will add to Section 2.2.1:

"We adopt adversarial training because: (1) It models complex fMRI-EEG joint distributions without parametric assumptions; (2) The discriminator’s FC-guided realism check enforces neurobiological plausibility;

Summary: GANs are indispensable for FEH because they:

1. Capture complex multimodal distributions beyond deterministic/VAE capabilities.
2. Denoise via adversarial filtering.
3. Enforce neurologically grounded hyperedges through FC-based discrimination.
4. Synergize OFEI-IHEN under a unified objective.

Q3: The paper does not discuss the limitations of the proposed method or provide insights into potential future extensions. Including such a discussion would enhance the completeness of the paper and help better understand its applicability and scope of application.

A3: We sincerely thank the reviewer for highlighting the need to explicitly discuss the limitations and future extensions of our work. This is a valuable suggestion that will strengthen the paper's completeness and contextualize our contributions. In the revised manuscript, we will add a dedicated "Limitations and Future Work" subsection (Section 4.1) with the following content:

---

4.1 Limitations and Future Work

While FE-NET demonstrates significant advantages in joint fMRI-EEG modeling, we acknowledge several limitations:

1. Computational Complexity: The integration of GAN-based hypergraph generation (FEH) and Neural ODEs (FED) incurs higher training costs than traditional GNNs. Future work will explore model distillation techniques to reduce inference latency without sacrificing performance.
2. Data Heterogeneity: Our experiments used the AAL-90 atlas for ROI alignment. Generalizability to finer-grained parcellations or datasets with varying MRI resolutions requires further validation.
3. Temporal Granularity: Although Neural ODEs mitigate sampling-rate discrepancies, modeling ultra-rapid EEG oscillations (<100ms) remains challenging. We plan to incorporate spike-based encoding for sub-second dynamics.
4. Clinical Translation: Current validation focused on healthy cohorts (LEMON/CN-EPFL). Future studies will test FE-NET on clinical populations (e.g., epilepsy, Alzheimer’s) to assess diagnostic utility.

Future Work:

• Cross-Modal Imputation: Extend FE-NET to synthesize missing EEG/fMRI data via latent-space interpolation, enabling single-modality inference.
• Dynamic Hypergraph Sparsification: Develop attention-based edge pruning to optimize incidence matrices during training, reducing memory overhead.
• Federated Deployment: Adapt the framework for privacy-preserving multi-site data integration using differential privacy.

---

These additions will clarify the boundaries of our method’s applicability while highlighting impactful research trajectories. We are grateful for the opportunity to enhance the paper’s rigor and translational relevance.

Comment:

We sincerely appreciate your meticulous review of our manuscript. Your valuable suggestions have helped a lot in improving the Clarity and Rigor of our work.

Q1: While there are many different components of the proposed approach, including GANs, hypergraphs, and Neural ODEs, the time complexity for this approach should be relatively high

A1: We conducted an Algorithm Complexity and computation resources Analysis for FE-NET.

At the same time, we provided a comparison of computational resources with other SOTA methods. Although FE-NET requires more computational resources than existing SOTA methods, The downstream tasks of fMRI and EEG imaging are not test-intensive, the slightly longer inference time after model training is completed will not interfere with the actual application of the model.

In fact, when the batch size is adjusted to 8, our model can run perfectly on a single 2080ti. Existing larger fMRI datasets are typically at the thousand-level scale, and even if thousand-level fMRI-EEG datasets emerge in the future, four 2080ti GPUs can handle them perfectly. Considering the cost of acquiring fMRI and EEG data, four NVIDIA 2080ti GPUs is a small price to pay. Given the considerable potential for future improvements in GPU performance, our model remains well-suited for large-scale datasets.

Time Complexity Analysis

FEH Module (GAN-based Hypergraph Generation)

• OFEI Algorithm: O(n²·m) where n=number of ROIs (typically 90-120), m=hyperedges per subject.
• IHEN Layers: O(L·(Nv·d² + Ne·k²)) where L=5 layers, Nv=vertices, Ne=hyperedges, d=feature dim (256), k=hyperedge cardinality.

FED Module (Neural ODE Embedding)

• Control Step: O(T·(Nv+Ne)·d²) where T=integration steps (~50). Dual-stream LSTM with attention dominates computation.
• Diffusion Step: O(Nv² + Ne²) per step from hypergraph Laplacian operations. Sparse matrix optimizations reduce practical complexity.

GPU Memory Usage

FE-NET runtime on LEMON Using 1 NVIDIA A6000

Training Performance

Inference Performance

Table: Computational Resource Comparison using NVIDIA A6000

Q2: There is no related work section. The authors should provide a comprehensive background introduction towards related work, such as employing ODEs for brain network analysis[1], etc.

A1: We thank the reviewer for this essential suggestion. We acknowledge the need to rigorously position FE-NET within the field and will add a dedicated subsection (1.2: Related Work). Below is our revision strategy to address this gap:

1. New Section 1.2: "Related Work on Multimodal Fusion and Asynchronous Modeling"

Structure and key additions:

1. Synchronous fMRI-EEG Fusion:

   "Early fusion methods required strict temporal synchronization. Advanced deep learning approaches improved synchronization through attention mechanisms but remained limited to temporally aligned data. These cannot handle real-world asynchronous acquisitions."

2. Asynchronous Modeling Attempts:

   "Some studies used kernel methods to align asynchronous data post-hoc but lost dynamic interactions. Others employed self-supervised techniques but treated modalities independently without joint topological modeling. Prior work has not resolved asynchronous fusion at the graph-structure level."

3. Neural ODEs in Neuroimaging:

   "Neural ODEs have modeled single-modality dynamics in fMRI data. Spatio-temporal extensions capture dynamic patterns but ignore cross-modal asynchrony. None have adapted ODEs for multimodal hypergraph evolution."

4. Hypergraph Neuroimaging:

   "Existing hypergraph neural networks modeled fMRI data but omitted EEG integration. Other works fused MRI modalities via hypergraphs but required temporal alignment. FE-NET is the first to leverage hypergraphs for asynchronous fMRI-EEG integration."

---

2. Explicit Novelty Positioning

FE-NET’s breakthroughs:

---

3. Addressing the Asynchronous Modeling Gap

• Temporal Flexibility:

  "Unlike methods that process modalities separately, our Neural ODE-based FED module continuously aligns fast EEG and slow fMRI dynamics within a unified latent space."

• Structural Innovation:

  "While existing approaches aligned async data through static methods, FE-NET’s hypergraphs encode many-to-many cross-modal interactions, preserving complex neurodynamics."

---

4. Revisions Commitment

In the manuscript:

• Add ~400 words in Section 1.2 (preceding the "Our Contributions" paragraph)
• Include key references covering:
  • Synchronous and asynchronous fusion techniques
  • ODE applications in neuroimaging
  • Hypergraph-based neuroimaging studies
• Novelty statement:

  "To our knowledge, this is the first framework to simultaneously resolve: (a) Asynchronous fMRI-EEG fusion via hypergraph-structured Neural ODEs, (b) GAN-based generation of multimodal hypergraphs, and (c) Dynamic embedding of cross-modal interactions without temporal resampling."

Summary: This revision will:

1. Formally establish FE-NET’s pioneering role in asynchronous fMRI-EEG hypergraph fusion
2. Provide scholarly grounding for technical innovations
3. Explicitly contrast with prior approaches to highlight topological and temporal advantages

We affirm the revised manuscript will comprehensively contextualize our contributions within the field.

Comment:

We sincerely appreciate your meticulous review of our manuscript. Your valuable suggestions have helped a lot in improving the Clarity and Rigor of our work.

Q1: Revise and improve claims of originality with better situation in the state of the art - will improve rating

A1: In accordance with your suggestions, we will amend the following imprecise expressions in the thesis.

1. Motivation Correction (Line 34)

• Original:
  "...simultaneous recorded fMRI-EGG dataset suffers from tiny dataset size (less than 20)..."
• Revised:
  "The sample size of simultaneously recorded fMRI-EGG datasets is markedly smaller than that of asynchronously acquired datasets, thereby limiting the reliability of extant research relying on synchronous recordings."

2. Field Characterization (Line 3)

• Original:
  "joint modeling of fMRI and EEG is a rarely explored area" that "has not yielded satisfactory results"
• Revised:
  "Deep learning-based joint modeling of asynchronously recorded fMRI and EEG is a relatively underexplored area and has not yielded satisfactory results."

3. Originality Claims

• Line 58 Revision:

  • Original:
    "this is the first attempt to model asynchronous EEG and fMRI data"
  • Revised:
    "This is the first attempt to model asynchronous EEG and fMRI data as Neural ODEs based hypergraph"

• Line 322 Revision:

  • Original:
    "For the first time, we attempt to model asynchronous fMRI and EEG data..."
  • Revised:
    "For the first time, we attempt to model asynchronous fMRI and EEG data as Neural ODEs based hypergraph."

Q2: In the discriminator loss function, why take FC edges as reference for "real" paths rather than EC edges? This seems like the discriminator will push the generator to compute multimodal connectivity matrices which look more like fMRI connectivity than EEG connectivity.

A2: Our decision to treat the fMRI‐derived FC matrix as the “real” paths in the discriminator is partially because our use of the EEG‐derived EC matrix in Equation 4. More specifical reasons are:

1. FC as a Stable, Gold-Standard Prior. The FC matrix, estimated over seconds‐long BOLD time series, offers a relatively low‐variance, anatomically plausible network “backbone.” Using FC in the adversarial loss provides the generator with a consistent, robust target distribution.

2. EC to Inject Electrophysiological Detail. By computing the initial hyperedge attributes $X_{k, E}^{(0)}$ from the EC matrix, we ensure that fine‐grained EEG dynamics are explicitly encoded in the generated hypergraphs. This EC initialization drives the generator to learn electrophysiologically meaningful interactions.

We swapped FC for EC in the discriminator’s “real” paths. which leads to a longer training time, and the average classification accuracy dropped by 2.8 %. This confirms that our original adversarial criterion is not merely a “balancing hack,” but a necessary mechanism to stabilize training and preserve both modalities’ contributions.

Q3: Provide improved description of how EEG and fMRI data are taken as input - will improve rating. The transition in Section 2.2.1 needs better framing for EEG/fMRI dimensionalities. The EEG montage highlights 30 electrodes (of 62), while fMRI ROIs are unspecified. Section 2.2.2 uses dimension $N$ for both modalities, implying 1:1 node correspondence without clarifying alignment. This must be explicitly addressed.

A3: We thank the reviewer for this essential critique. The lack of clarity regarding node alignment will be rectified. Below, we provide clarification and outline revisions:

1. Node Alignment Strategy

We confirm explicit 1:1 anatomical correspondence** between EEG and fMRI nodes via standardized neuroimaging pipelines:

• fMRI parcellation: All subjects use the AAL-90 atlas (90 cortical/subcortical ROIs).
• EEG processing:
  • Full 62-electrode data undergo source localization (sLORETA) to cortical voxels.
  • Voxel-level signals are spatially aggregated to the same 90 AAL ROIs as fMRI.

Clarification: The "30 red electrodes" in Figure 1 were purely for visual emphasis of key regions. All 62 electrodes were used for analysis. We will revise the figure to eliminate ambiguity.

2. EEG Source Localization Details

The pipeline is:

1. Forward modeling: Boundary element method (BEM) with standard head model.
2. Inverse solution: sLORETA for source reconstruction.
3. ROI aggregation: Voxel-level source signals averaged within each AAL-90 region.
4. EC computation: Partial directed coherence (PDC) between ROI-level time series.

Supplementary Material will includes:
"EEG signals were source-localized to AAL-90 ROIs via sLORETA . ROI-level bandpower features and PDC-based EC matrices were then computed."

3. Revisions Commitment

We will:

1. Add a new subsection (2.1.1: Data Alignment and Node Correspondence):

2. Update Figure 1:

   • Replace the "30 highlighted electrodes" with a schematic of 62 electrodes → source localization → AAL-90 ROIs.

---

Summary: Our framework relies on anatomically grounded 1:1 node correspondence achieved through EEG source reconstruction and fMRI parcellation. We apologize for the ambiguity in the original manuscript and thank the reviewer for highlighting this gap.

Q4: If these initial hypergraphs {G′k} are directly utilized in the generative process to achieve multimodal connectivity, the robustness of the resultant generation may be compromised.", but why is that? To me this is lacking a justification.

A4: We appreciate this critical observation and acknowledge the need for explicit justification. Below, we clarify the core reasons:

---

1. Subject-Specific Noise Amplification

• Problem:
  Initial hypergraphs {G'ₖ} are constructed per subject using Dynamic Hypergraph Construction (DHC). DHC:
  • Sensitively encodes subject-level noise/artifacts (e.g., motion in fMRI, muscle artifacts in EEG).
  • Lacks cross-subject consistency, leading to fragmented topological representations.
• Consequence:
  Feeding noisy {G'ₖ} directly into the GAN generator propagates subject-level biases, degrading multimodal fusion.
• OFEI Solution:
  Computes an optimally isomorphic hypergraph $G$ (Eq. 3) that maximizes group-level similarity:

$

G = \operatorname*{argmax}{G}\sum{k}\text{Sim}(G,G'_k)

$

This denoises by extracting the most consistent topological "backbone" across subjects.

---

2. Modality-Specific Structural Bias

• Problem:
  {G'ₖ} are built separately for fMRI and EEG using modality-specific heuristics in DHC:
  • fMRI hyperedges model spatial communities (e.g., ROI clusters).
  • EEG hyperedges model spectral-temporal interactions.
• Consequence:
  Direct concatenation (e.g., $G_k = G'_{\text{fMRI},k} \| G'_{\text{EEG},k}$) creates modality-centric hypergraphs with conflicting structures, inhibiting cross-modal information flow.
• OFEI Solution:
  Forces fMRI and EEG into a shared isomorphic space via Eq. 2–3, ensuring:
  • fMRI hemodynamic patterns and EEG oscillations co-evolve in aligned hyperedges.
  • Biologically implausible cross-modal connections (e.g., unrelated ROIs) are pruned.

---

Q5: Section 2.2.2 mentions the bold time series B is "employed to characterize the nodal attributes" (X_{k_V}) - how? what is used from the B time-series? GNNs are very sensitive to node feature choice and their scaling, so it would be useful to specify.

A5: the BOLD time series $B$ is employed to derive nodal attributes $X_{k,V}$ through the following process:

• We extract the mean BOLD signal from each of the predefined regions of interest (ROIs) corresponding to the AAL atlas.
• Specifically, for each ROI, we compute the mean, standard deviation, and temporal correlation across time points to serve as features.
• To ensure robustness against scaling issues, we apply standardization (zero mean, unit variance) to all nodal features before feeding them.

Q6: It is not clear why fMRI is used for initial node attributes and EEG for hyperedge attributes. Could we swap EEG and fMRI in the inputs?

A6: Our assignment—fMRI for node attributes and EEG for hyperedge attributes—is grounded in neurophysical principles. Below, we justify this design:

---

1. Neurobiological Alignment

• fMRI → Node Attributes:
  • fMRI's BOLD signals reflect slow hemodynamic changes (seconds-scale) localized to specific anatomical regions (AAL ROIs).
  • This naturally characterizes static regional properties (e.g., activation magnitude, variance), ideal for node-level features.
• EEG → Hyperedge Attributes:
  • EEG captures millisecond-scale oscillations with high temporal precision but low spatial resolution.
  • Hyperedges represent functional ensembles—making EEG's interaction metrics perfect for hyperedge-level features.

---

2. Hypergraph Semantic Consistency

• Neurobiological interpretation:
  • A hyperedge connecting {ROI₁, ROI₂, ROI₃} signifies "these regions co-activate with timing captured by EEG."
  • Assigning fMRI to hyperedges would conflate hemodynamic correlation (slow) with neural interaction (fast), violating biophysical principles.

Summary: Our modality-to-attribute mapping:

1. Respects neurophysical realities (fMRI=spatial, EEG=temporal).
2. Ensures hypergraph interpretability (nodes=regions, edges=EEG-guided interactions).

Our additional experiments also show swapping modalities would degrade performance. We thank the reviewer for prompting this fundamental clarification.

Comment: We sincerely appreciate your meticulous review of our manuscript. Your valuable suggestions have helped a lot.

Q1.1: Given the complexity introduced by GANs, Neural ODEs, and hypergraphs, could you provide a detailed computational complexity analysis?

The computation complexity analysis is shown below:

Time Complexity Analysis

FEH Module (GAN-based Hypergraph Generation)

• OFEI Algorithm: O(n²·m) where n=number of ROIs (typically 90-120), m=hyperedges per subject.
• IHEN Layers: O(L·(Nv·d² + Ne·k²)) where L=5 layers, Nv=vertices, Ne=hyperedges, d=feature dim (256), k=hyperedge cardinality.

FED Module (Neural ODE Embedding)

• Control Step: O(T·(Nv+Ne)·d²) where T=integration steps (~50). Dual-stream LSTM with attention dominates computation.
• Diffusion Step: O(Nv² + Ne²) per step from hypergraph Laplacian operations. Sparse matrix optimizations reduce practical complexity.

GPU Memory Usage

FE-NET runtime on LEMON Using 1 NVIDIA A6000

Training Performance

Inference Performance

Q1.2 Could you include an analysis comparing training time, inference speed, and GPU usage with existing baseline methods?

A1.2: we provided a comparison of computational resources with other SOTA methods. Although FE-NET requires more computational resources than existing SOTA methods, The downstream tasks of fMRI and EEG imaging are not test-intensive, the slightly longer inference time after model training is completed will not interfere with the actual application of the model.

In fact, when the batch size is adjusted to 8, our model can run perfectly on a single 2080ti. Existing larger fMRI datasets are typically at the thousand-level scale, and even if thousand-level fMRI-EEG datasets emerge in the future, four 2080ti GPUs can handle them perfectly. Considering the cost of acquiring fMRI and EEG data, four NVIDIA 2080ti GPUs is a small price to pay.

Table: Computational Resource Comparison using NVIDIA A6000

Q3: Potential issues such as overfitting, dependency on high-quality preprocessing, and uncertain generalization to clinical or pathological populations are currently not discussed. Could you include a dedicated section addressing these points?

A3: Thank you for highlighting these important aspects. We plan to add a new section entitled “Limitations, Robustness and Future Directions” that addresses these points:

1. Overfitting

• In this subsection, we will describe our strategies for mitigating overfitting, including k-fold cross-validation, early stopping based on a held-out validation set, L₂ weight regularization, and dropout layers.
• We will report learning curves and validation losses to demonstrate convergence without divergence between training and validation performance.

2. Generalization to Clinical and Pathological Cohorts

• We clarify that all current experiments use healthy adult datasets, and that population shifts may impact model behavior.
• We outline a planned evaluation on two other clinical cohorts. We emphasize that understanding failure modes in these groups is a priority for future work and welcome collaborations for data sharing.

Q4: It is unclear if your model handles common practical issues like missing EEG channels or varying time windows. Could you provide relevant tests or a discussion addressing these real-world challenges?

A4: Below, we clarify how FE-NET could address these challenges and plan to perform additional experiments to validate its robustness:

1. Handling Missing EEG Channels

Architectural Robustness:

• FE-NET models fMRI-EEG data as hypergraphs, where hyperedges (unlike simple edges in graphs) connect multiple nodes (ROIs). This structure can mitigates missing channels:
  • If an EEG channel (node) is missing, information can propagate via shared hyperedges connecting other nodes in the same functional group.

Empirical Validation:
We will conduct ablation studies on the LEMON dataset by:

• Randomly masking 10–30% of EEG channels during inference.
• Measuring performance decay (accuracy, AUC) on attention/intelligence classification tasks.

---

2. Varying Time Windows

Neural ODEs for Irregular Sampling:

• The FED module leverages Neural ODEs (Eq. 13–16) to model continuous dynamics, making it intrinsically suitable for irregular/inconsistent time windows:
  • Neural ODEs interpolate latent states between arbitrary timestamps, accommodating variable-length EEG segments without resampling.
  • The Lie-Trotter discretization (Eq. 16) decouples control (modality-specific dynamics) and diffusion (cross-modal interactions), allowing asynchronous updates for fMRI (slow) and EEG (fast) streams.

Experimental Verification:
We will test FE-NET on non-uniform time windows by:

• Segmenting the LEMON EEG data into variable intervals (e.g., 2s, 5s, 10s windows).
• Evaluating classification consistency across window lengths.
• Benchmarking against RNN/LSTM-based models (which require fixed-length inputs).

---

Q5: The paper lacks insight into what temporal representations are learned by the Neural ODEs. Could you provide an analysis or visualization explaining what dynamics the Neural ODEs capture across modalities and their contribution to classification?

A5: We appreciate this insightful feedback and acknowledge that further analysis of the temporal dynamics learned by Neural ODEs would strengthen the paper. Below, we propose a analysis approach to address this gap:

1. Temporal Dynamics Captured by Neural ODEs

The FED module uses Neural ODEs to model the latent evolution of joint fMRI-EEG representations. Key learned dynamics include:

• Cross-Modal Synchronization: Neural ODEs reconcile fMRI’s slow hemodynamic changes (seconds) and EEG’s rapid oscillations (milliseconds) by learning a shared latent differential equation: $$\frac{d}{dt}\begin{bmatrix} \mathbf{X}_v(t) \\ \mathbf{X}_e(t) \end{bmatrix} = f\left( \begin{bmatrix} \mathbf{X}_v(t) \\ \mathbf{X}_e(t) \end{bmatrix}, t \right)$$ Here, $f(\cdot)$ captures inter-modal dependencies (e.g., how transient EEG spectral power fluctuations correlate with delayed fMRI BOLD responses in specific ROIs).
• Asynchrony Handling: The ODEs intrinsically align mismatched sampling rates by interpolating latent states continuously across time, avoiding manual synchronization.

2. Proposed Analysis & Visualization

To elucidate these dynamics, we will:

• Visualize Latent Trajectories: Plot the evolution of vertex/hyperedge features ($\mathbf{X}_v(t)$, $\mathbf{X}_e(t)$) for exemplar subjects (Figure 1, right). We will highlight:
  • Convergence points where fMRI and EEG representations stabilize (indicating synchronized cross-modal states).
  • Derivative hotspots ($\frac{d\mathbf{X}}{dt}$) during task-relevant intervals.
  • Ablation with Fixed Intervals: Replace Neural ODEs with RNNs/LSTMs (using fixed timesteps) and compare performance degradation.

Q6: Although the LEMON dataset is sizable, reliance on it and the very small CN-EPFL dataset limits generalization claims. Could you include experiments using additional datasets to strengthen claims of robustness.

A6: We further conducted performance comparisons on the CWL and Oddball datasets, the result is shown in Supplementary materials. Our model still shows advantages on these datasets. We also planned do evaluation on other clinical cohorts. We emphasize that understanding failure modes in these groups is a priority for future work and welcome collaborations for data sharing.