BrainCodec: Neural fMRI codec for the decoding of cognitive brain states

The following 6 questions correspond to Weaknesses.

1). Individual Application: Since fMRI provides valuable insights into functional impairments faster than structural imaging, an effective pipeline for individual fMRI signal analysis and decoding would be highly beneficial. The authors appear to have developed a novel framework similar to AOE, which typically requires a large dataset for training, even in unsupervised learning settings. However, the authors do not discuss the applicability of their framework to individual fMRI data. If it cannot be applied on a personal level, the framework’s clinical translational potential may be limited.

2). The Concept of Codebooks: The authors applied their framework to both resting-state and task-based fMRI to establish a codebook explaining neural activity. However, they provide only qualitative explanations and visualizations. Reviewers suggest that the authors could strengthen the connection between the codebook and established fMRI components, such as Brain Connectivity Networks (BCNs) or specific task paradigms. The reviewers would like to see the codebook serve as an alternative technique for representing brain activity. For example, task-codebooks could be compared with task paradigms and be linked with spatial features, like BCNs, to clarify their role in fMRI analytics. Importantly, reviewers are particularly interested in understanding whether codebooks could contribute to early diagnostics for neurological disorders, such as Alzheimer’s Disease (AD) and Traumatic Brain Injury (TBI). Meanwhile, the codebook concept should be introduced and contextualized in the introduction section.

3). Hyperparameter Tuning: It appears that essential hyperparameters in this work were determined manually. Reviewers are concerned that some hyperparameters, such as the number of artificial neurons, could significantly impact the Braincodes's outcomes. A detailed description of the hyperparameter tuning process would enhance transparency and reproducibility.

4). Optimization: The authors presented a complex model but did not provide sufficient details about the optimization approach used in this work. Reviewers would like clarification on the choice of advanced gradient-based optimizers (e.g., Adam, STORM, or SGD), along with specifics on the learning rate and other vital parameters.

5). Datasets: The authors did not include detailed demographic information on the datasets used in their experiments. Reviewers recommend providing specifics on the number of subjects, as well as age and gender distributions.

6). Empirical Experimentation: Reviewers found the experimental design somewhat confusing. In section 4.2.1, the authors report codebooks from resting-state fMRI, yet multiple task-based codebooks are mentioned within this section. Reviewers suggest that results from resting-state and task-based fMRI should be reported separately, as these data types are distinct. Additionally, the authors only compare the reconstructed fMRI data using BrainCodec with BrainVAE. Including comparisons with other representative linear learning methods, such as Independent Component Analysis (ICA) and Sparse Dictionary Learning (SDL), would allow for a more comprehensive assessment of BrainCodec’s time efficiency and reconstruction accuracy.

1. Limited Novelty and Contribution: The paper fails to clearly distinguish the contributions of BrainCodec from existing methods, particularly those discussed in Thomas et al. (2022), which include linear approaches and Causal State Modeling (CSM). The authors should articulate the specific advancements that BrainCodec offers in relation to prior work to establish its significance.

2. Inadequate Comparison with Established Techniques: The evaluation lacks a thorough comparison with state-of-the-art techniques in fMRI analysis. A robust comparative analysis is essential to validate the proposed method’s effectiveness and situate it within the current landscape of fMRI processing methodologies.

3. Insufficient Methodological Details: The paper does not provide adequate details about the experimental setup, especially regarding the processing and parcellation of fMRI data into regions of interest (ROIs). A clearer methodology is crucial for reproducibility and understanding the implications of the findings.

• The performance of BrainCodec seems to vary across different datasets, with the MDTB dataset showing less improvement compared to the HCP and Over100 datasets. The authors acknowledge the dataset-specific characteristics, but further investigation or more discussion into the factors affecting performance would be beneficial.
• For Table4, besides of L1 dist, spatial correlation could also be a potential metric from a different aspect to evaluate the similarity between two patterns, which might provide a more comprehensive evaluation.
• What would happen if the codebook number continue increasing, like to 12, 16, etc.

How does BrainCodec compare to other fMRI compression or denoising techniques like temporal ICA or sparse dictionary approaches?

How sensitive is the method to the choice of hyperparameters, particularly the number of codebooks?

What are the computational requirements and processing time for BrainCodec?

Different cognitive tasks have different durations/stimuli blocks. Would sampling all tasks to 50 timepoints disproportionately affect certain task types?

Could the matching of TRs to a predetermined range lead to slight misalignments between the actual timing of fMRI data and its representation in the model? I am having hard time wrapping my head around this.