Evaluating topological fitness of human brain-inspired sub-circuits in Echo State Networks

One of the claims in the paper is that it “found that the performance of the reservoir model is not strictly defined by the reservoir’s echo-state property,” but it incorrectly associates this with “the spectral radius of the connectivity matrix.” This is a misunderstanding, as it is commonly stated that if the spectral radius of the connectivity matrix is below 1, the network respects the echo-state property. However, a network can still respect the echo-state property even if its spectral radius exceeds 1. This confusion undermines one of the paper’s key claims.

Additionally, the concept of memory capacity is incorrectly attributed to Suarez et al. (2020), when it should be credited to Jaeger (2002). The paper also misuses this concept. To clarify, Suarez (2020) does cite Jaeger (2002), so this mistake is not theirs.

The structure and wording of the paper is also confusing. For instance, the section titled “Structural and functional connectomes” should not be part of the dataset description, but rather included in the section explaining how the models are formed. Referring to a model as "control" is not common in Machine learning.

The results are too weak to be really interesting as many previous paper (rightfully cited in the present one) have already explored this path.

1 The Default Mode Network (DMN) is a resting-state network and may not optimally support task-specific performance enhancements. To address this, the authors could include a rationale for using DMN in active task contexts or, preferably, conduct comparisons with task-relevant networks like the Executive or Attention Networks. Testing these would clarify if the observed performance gains are unique to DMN topology or simply due to network modularity.

2 The paper lacks comparisons with empirical neurobiological data, which would strengthen the biological plausibility of embedding functional networks in ESNs. Incorporating neural data, such as functional MRI or EEG data for verification, or comparing modeled task performance with observed human data would improve the paper's rigor and relevance to neuroscience.

3 The effects of key parameters (e.g., connection density, node centrality) on ESN performance are not fully explored. Conducting sensitivity analyses or providing ablation studies on these parameters could help understand which aspects of DMN topology are crucial for performance improvements.

4 The study only compares DMN-based ESNs with random networks, without considering other biologically plausible alternatives like small-world or scale-free networks. Including these structures would provide a broader understanding of which network topologies, beyond DMN, contribute to the observed performance gains.

5 While the study highlights improvements on experimental tasks, its implications for real-world or long-term applications remain unclear. Testing the ESN on more complex, real-world-inspired tasks or applications would offer insights into the practical relevance and robustness of the proposed approach.

1. There are some missing related works. For example, Ref[1] performs a similar analysis with RNN, Ref[2] talks about modularity with HebbFF, and Ref[3] summarizes the potential paradigms of integrating connectome to ANNs. Relating the current work with existing ones will help position the value of the current work.

2. Why do you evaluate according to the 7 readout Yeo network? Although this one is popular, it needs verification about the robustness of the results here with other potential partitions. In addition, considering the number of neurons in different networks are distinct, it is necessary to control the covariants here.

3. Many of the results, including but not limited to those in Figure 3,4,5 and Table 1 are reported with the statistical significance. Thus it is hard to tell whether the differences are statistically meaningful. Some perturbation and bootstrap based statistics should be reported here to improve the quality of the results.

4. The results in Table 1 is quite isolated with the others. The authors should focus on the cognitive tasks and provide some mechanistic insight on what kind of specific topology or property in the connectome leads to the improvement of model performance.

5. The relationship between network measurement and model performance could be investigated across subjects as well where you can adopt the FC/SC for each subject separately and evaluate their performances on tasks. HCP or other dataset with task data would be helpful here.

Ref: [1] https://www.nature.com/articles/s41467-022-28323-7 [2] https://www.science.org/doi/full/10.1126/sciadv.adm8430 [3] https://www.nature.com/articles/s41586-023-06670-9

The contribution of this paper feels limited, as it neither introduces novel methodological advancements nor offers in-depth insights into the experimental results. The study primarily involves modifying network structure, observing performance variations, and drawing conclusions. Given the study's focus on computational neuroscience, providing theoretical explanations or a mathematical basis for the observed phenomena would be essential to strengthen the findings and their broader applicability.