Professor X: Manipulating EEG BCI with Invisible and Robust Backdoor Attack

Questions

What motivated the specific design choices made in the three-stage clean-label poisoning strategy?

Thanks for your interest in the design of Professor X! Actually, we design our attack in a reverse order, that is, we (1) firstly design the frequency injection stage, （2） then design the reinforcement learning stage, (3) lastly design the trigger selection from true dataset to do clean label attack. With this reverse order, you can better understand the motivation behind our design, and you will find the design of Professor X is very intuitive.

1. We firstly notice that all backdoor attacks for time-series and EEG BCI are injecting triggers in the temporal domain, which will inevitably bring unnatural frequency into the frequency domain. Thus, we pioneerly propose to inject triggers in the frequency domain. We combine two data's amplitude in the frequency domain to generate poisoned data. Since these two data are all natural data, the generated poisoned data is natural no matter in the temporal or frequency domain. We visualize the poisoned data and clean data in Fig 2 (frequency domain) and Fig 8 (temporal domain) to verify the stealthiness.

2. Now we know that injecting trigger in the frequency domain is stealthy and natural, but we don't know which electrodes or frequencies should we inject the trigger. Since for different EEG tasks, the EEG electrodes and frequencies strongly related to the performance of EEG BCI are different. For the first time, we propose to adopt some optimization algorithm for leaning the injecting strategy. After lots of experiments, we chose the reinforcement learning as our optimization algorithm. Moreover, we design two novel losses (DIS and HF) to enhance the stealthiness and robustness.

3. Last but not least, we have ensured the performance of Professor X, but we notice that certain classes of EEG data have specific morphology that can easily be identified by human experts, like the EEG during the ictal phase contains more spike/sharp waves than those during the normal state phase. In short, different classes of EEG data have different frequency distribution. To enhance the stealthiness, we introduce the clean label attack. The poisoned data has the same label as the trigger data, so the poisoned data's frequency information dosen't contain any frequency distribution from other classes.

Consequently, we design our novel Professor X, a three-stage clean-label poisoning attack.

How does the proposed method compare in effectiveness to existing backdoor attack methods in the literature?

We compare our attack with four baselines in our experiments, the results are displayed in Table 2 and Fig 3. We detail the experiment settings in section 4.3. Specifically,

1. We generate the poisoned data with different backdoor attack methods, and mixing these poisoned data into a bigger clean training dataset to form a poisoned dataset.

2. We train an EEG BCI model on the poisoned dataset, the trained EEG BCI model is injected with a backdoor, we call it backdoor model.

3. We test the backdoor model on the clean test set, and the poisoned test set where we inject triggers into the clean data. We report the clean accuracy and attack success rate by running this experiment.

Except for the different methods used to generate poisoning data, the other experimental setups are all same. By this experiment we compare our attack with previous baselines.

---

We hope this response could help address your concerns. We believe that this work contributes to this community and has the potential to serve as a catalyst for its development. We would sincerely appreciate it if you could reconsider your rating and we are more than happy to address any further concerns you may have. Thanks again!

Thanks for your time and efforts in reviewing our work! Your questions about the motivation of our attack's design is insightful, which also provide an opportunity to summarize our contributions. Below we carefully address them.Below we have addressed your questions and concerns point-by-point.

Weaknesses

The literature review is notably limited ... A more thorough engagement with relevant studies would enhance the study's contributions and contextual relevance.

Thanks for providing the additional references and we have added them in the revised paper, please kindly refer to the new version of our paper. However, we would like to kindly clarify that paper [1,2] are about adversarial attack on EEG BCI models, which is not the same as backdoor attack. In the revised paper, we have clarified the difference.

Although backdoor attack and adversarial attack are all about the vulnerability of deep models, backdoor attack is different from adversarial attack in two ways: (1) Attacking phase: while adversarial perturbation attacks the model in the inference phase, backdoor attack injects a backdoor in the training phase; (2) Attacking objective: while adversarial attack aims to let deep models misclassify (the attacker doesn't care about the target class models will misclassify), backdoor attack aims to let deep models misclassify the samples with particular triggers to target class (the attacker clearly knows the target classes models will misclassify, thus can manipulate the model's output by injecting different triggers).

Moreoever, the adversarial perturbation can also be used as a trigger for backdoor attack, which has been researched in [3]. In our paper, we have compared [3] (the baseline AdverMT) at the multi-trigger and multi-target settings in our paper. Please kindly refer to Table 1, it can be observed that our attack outperforms the adversarial-based backdoor attack. As the adversarial perturbation is designed for single-target attack, it fails to attack multi-target classes.

In conclusion, we would like to kindly argue that the overlook of two less relevant papers [1,2] won't diminishes the impact and originality of our research. We sincerely hope you can reconsider your score.

[1] X Zhang, et al. "On the vulnerability of CNN classifiers in EEG-based BCIs", IEEE TNSRE, 27(5):814–825, 2019.

[2] Z Liu, et al. "Universal adversarial perturbations for CNN classifiers in EEG-based BCIs", 2021.

[3] L. Meng, et al. "Adversarial filtering based evasion and backdoor attacks to EEG-based brain-computer interfaces", Information Fusion, 2024.

Thank you for your thoughtful and insightful suggestions! We believe we have comprehensively addressed your questions regarding relative literature, Professor X's specific design, its effectiveness compared to other baselines including adversarial-based attacks. 

We would like to emphasize that our method is the first to achieve both highly stealthy and robust backdoor attacks on EEG BCI. Through data poisoning approach, our method even does not require controlling the training stage of target models.   

We are wondering whether you have any additional questions or comments regarding our response to your review comments. We will do our best to address them.   

We sincerely appreciate the time and effort you have dedicated to reviewing our manuscript. Thank you for your thoughtful consideration!

Dear Reviewer dg1E:

Thanks for your valuable time in reviewing our paper! We are writing this comment to kindly request for post-rebuttal comments.

We have added the references you provided into out revised paper, and clarify the difference between our work and these references. Have we already addressed your concern? Or do you have any further concerns?

Please feel free to contact us, we are more than happy to hear from you. We would really appreciate if you could reconsider your rating and we are glad to address your further concerns.

Best,
Authors

We truly thank you for your appreciation of our work! Your question about how to defend Professor X is important and meaningful, which also helps us deepen the understanding of our Professor X attack. Below we have addressed your questions and concerns point-by-point.

Weaknesses

The potential for malicious use raises significant ethical issues, as manipulating EEG outputs could lead to harmful consequences for users and their applications.

We can not agree more! As we discussed in the general responses, the malicious use of our techniques raises severe ethical issuses. Unfortunately, we did not find any effective ways to defend Professor X due to the high stealthiness and robustness. We will try our best to find a method to guard our attack, and we here call for defensive research against Professor X.

However, we would like to kindly argue that regarding the high stealthiness and robustness of our attack as a weakness is not suitable. While our work is aiming to alert the EEG community of the potential hazard of backdoor attack, addressing the ethical issues raised by our attack is somewhat out of our paper's scope.

The method involves a sophisticated three-stage clean label poisoning attack, which may be complex to implement in practice, especially for those lacking expertise in reinforcement learning and EEG signal processing.

Thank you for rasing this concerns. We admit that our work is somewhat sophisticated, but every stage in our attack is meaningful and effective.

1. We propose to inject triggers in the frequency domain, which prevents from bring any unatrual frequency into the clean data.

2. We propose to adopt some optimization algorithm for leaning the injecting strategy to determine which electrodes or frequencies should we inject the trigger. Moreover, we design two novel losses (DIS and HF) to enhance the stealthiness and robustness.

3. We introduce the clean label attack to enhance the stealthiness since different classes of EEG data have different frequency distribution.

You can see the clear motivation behind the design of these three stages, as each of them improves the stealthiness or effectiveness of Professor X. However, we can still simplify our attack in the following ways:

1. Our methods can be applied without RL if some performance drop is acceptable. As displayed in Table 3, removing the RL will cause about 15% drop in Attack performance.

2. If you do not have to consider the different frequency distribution across different classes, then the trigger can be selected arbitrarily without clean label attack.

The design's focus on particular EEG tasks might result in overfitting, reducing its effectiveness in more generalized scenarios or with novel tasks.

We would like to take this opportunity to emphasize that we did not design our attack by focusing on any particular EEG tasks and our attack is generalizable across different EEG tasks, formats and EEG models. To verify the generalizability of our attack, we elaborately selected three EEG datasets as described in section 4.1. These datasets vary significantly in tasks, electrode numbers, montages, and sampling rates. The experimental results demonstrates the effectiveness across these different datasets, proving the generalizability.

From table 1, our method has been proven to be effective when facing various situations (diverse EEG models, tasks, and formats). Furthermore, we evaluate our attack on another public dataset which studies the P300 tasks [1,2]. The attack performances of three different EEG models on the dataset are still excellent:

It is worth mentioning that these results are obtained by only running the reinforcement learning 30 iterations, which takes only 0.5 hour on each model. These results can be another strong evidence to demonstrate the generalizability to other EEG datasets and real-world scenarios.

So we are confident that our attack will be still effective in more generalized scenarios or with novel tasks. Or maybe we misunderstand what you meant, if you have any questions, please ask us.

[1] U. Hoffmann, et al. "An efficient P300-based brain-computer interface for disabled subjects", J. Neurosci.Methods, 2008.

[2] Rodrigo Ramele. P300-Dataset. https://www.kaggle.com/datasets/rramele/p300samplingdataset

The approach may face challenges in scaling up for larger or more complex datasets, potentially limiting its effectiveness in real-world applications involving diverse user populations.

We would like to clarify that our method is effective no matter how large the size of EEG datasets. As the we poison the dataset by a constant poisoning ratio, the larger the target dataset, the more poisoned data we inject into the dataset.

Or the word "large" indicates the number of subjects in the dataset. However, our attack is still effecitve, as the experiments performed in our paper are all under cross-subject setting. We adopt the same experiment setup as that in the paper [3]. In short, we train the EEG model on the data from n-2 subjects, and test the EEG model on the data of a new subject. Our attack works very well under this cross-subject setting, so we can conclude that our method is effective no matter how large the number of subjects in the EEG datasets.

As for the word "complex", we kindly guess that does this mean the complexity of classification task? Like, the number of label types. If so, we would like to argue that our attack is effective at most situations ( > 90%). Because in most cases, the number of label types in EEG BCI is no mare than four. The datasets selected in our experiment are the most complex datasets. For emotion recognition, most datasets only focus on binary classification, like DEAP [4] and DREAMER [5]. For motor imagery, most datasets only do binary classification. As far as we know, the BCIC-IV-2a is the only dataset to do four-class classification. For epilepsy detection, most datasets only do itcal or non-itcal binary classification. We have refined the task to four categories.

From table 1 and the addtional experimental results on the P300 tasks above, we are quite confident that our attack will be effective facing challenges in scaling up for larger or more complex datasets.

[3] Meng L, et al. EEG-based brain–computer interfaces are vulnerable to backdoor attacks[J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2023.

[4] Koelstra S, et al. Deap: A database for emotion analysis; using physiological signals[J]. IEEE transactions on affective computing, 2011.

[5] Katsigiannis S, Ramzan N. DREAMER: A database for emotion recognition through EEG and ECG signals from wireless low-cost off-the-shelf devices[J]. IEEE journal of biomedical and health informatics, 2017.

Questions

What ethical frameworks are in place to govern the use of techniques like Professor X, and how can researchers ensure that such methods are not misused?

We can require professionals in the field of EEG BCI to sign relevant guarantees that they will not use any harmful attack, but afterall this is not the fundamental solution. Only by developing an effective defensive or detection method can we stop the misuse of such techniques.

Unfortunately, we did not find any effective ways to defend or detect Professor X due to the high stealthiness and robustness. To the best of our knowledge, we can not ensure that such methods won't be misused at this stage. We will do our utmost to find a way to protect our attacks. Here, we are calling for research on the defense against Professor X.

How effective is STRIP in detecting backdoor inputs under various levels of perturbation? Are there specific types of perturbations that significantly impact its performance?

STRIP detects the backdoor based on thees findings: 1) backdoor trigger is input-agnostic; 2) backdoor trigger is strong and effective when performing input perturbation; 3) The backdoor models' outputs (softmax) of poisoned data has very low entropy.

As the original STRIP paper [6] said, their detection method is insensitive to the trigger-size, so STRIP is effective no matter the trigger is big or small. In our understanding of STRIP, it can detect the trigger that is obvious to models in the original space (like temporal domain for time series data, and spatial domain for image data).

However, any trigger that may affect the above findings may cause STRIP's detection failure. For example, 1) the trigger is input-specific; 2) the trigger is not that strong, it fails when performing input perturbation; 3) The trigger won't cause the backdoor model to predict with very low entropy.

Our trigger is injected in the frequency domain, leading to the input-specific pattern in the temporal domain, causing finding 1 to be invalid. Moreover, the input perturbation in the temporal domain may damages the frequency information, causing our trigger disapper, leading to the finding 2 to be invalid. Last, as EEG is a nonstationary modality, the outputs of EEG models are always with high entropy, making finding 3 to be invalid. Thus, STRIP is not effective in detecting Professor X attack.

We hope our answer addressed your concerns, if you have any questions, please feel free to ask.

How does STRIP compare to other defense mechanisms in terms of robustness against backdoor attacks like Professor X? What are its relative strengths and weaknesses?

Thanks for you interest in our work since you ask such an insightful question. We evaluated several backdoor defensive method in our paper, including Neural Cleanse [7], STRIP [6], Spectral Signature [8], and Fine-Pruning [9].

STRIP has several strengths:

1) Insensitive to trigger-size: Neural Cleanse tries to detect backdoor by reconstructing the trigger. However, Neural Cleanse is sensitive to the trigger-size, while STRIP is effective no matter the trigger is big or small. STRIP can detect the trigger that is obvious to models in the original space (like temporal domain for time series data, and spatial domain for image data).

2) Plug and Play: Neural Cleanse needs the full control of the backdoor model as it inverse the trigger by adversarial loss. Spectral Signature needs the output of the last hidden layer, then perform singular value decomposition on the covariance matrix of these output. Fine-Pruning needs to detect the low-activated neurons and then prune them. While STRIP is plug and play, and compatible in any models. We only need the inputs and outpus of the backdoor models (treated as a blackbox as we don't need any intermediate outputs), then calculate the entropy of the outputs.

3) Backdoor model architecture-agnositc STRIP only needs the inputs and outputs of the backdoor model, so it is an architecture-agnositc method and is generalize to many real-world application senarios.

But as we discussed before, STRIP has some weaknesses:

1) STRIP can only detect backdoor whose trigger is input-agnostic, it fails when the trigger is input-specific. While Neural Cleanse and Spectral Signature can guard some input-specific backdoor triggers.

2) STRIP can only detect backdoor whose trigger is strong and still effective when performing input perturbation. It fails when the triggers disappear when performing input perturbation. While other backdoor defensive methods don't have this limitation.

3) STRIP can only detect backdoor whose trigger causes models predict with low entropy. If the model's outputs are always of high entropy, STRIP fails. While Fine-Pruning doesn't have this limitation.

What specific mechanisms within the model lead to the observed drop in attack success rate (ASR) when Fine-Pruning is applied? Can these mechanisms be quantified?

We are sorry that we don't have enough knowledge to perfectly answer this questions. The interpretation of EEG models is beyond our professional knowledge scope. But we try our best to share our understanding with you.

As far as we know, Fine-Pruning [9] removes neurons in the model that are in a dormant state (i.e., have low activation values) when predicting clean samples, thereby disrupting backdoor behavior. So the drop in ASR is caused by pruning the neuron that is important to predict poisoned data (this neuron has high activation values when processing inputs with triggers). But we don't know whether these mechanisms can be quantified or not.

How are low-activated neurons determined, and could this method inadvertently remove important features that are crucial for classification?

Fine-Pruning [9] assumes that the defender has a validation dataset $D_{valid}$ in which all data are clean. The defender feeds these clean data into the backdoor models, and recrods the average activation of each neuron. Afterwards, the defender iteratively prunes neurons from the DNN in increasing order of average activations. Thus, the low-activated neurons are those the average activation is low when feeding in clean data.

Yes, Fine-Pruning might inadvertently remove important neurons that are crucial for classification. Because the average activation is obtained from the small subset $D_{valid}$, so the low-activated neurons determined by $D_{valid}$ may be high-activated neurons when feeding another clean validation dataset $D_{valid}’$. That is, the important neurons for classifying clean sample $x \in D_{valid}’$ may be low-activated neurons for all samples in $D_{valid}$, resulting in the pruning of these important neurons.

How effective is Fine-Pruning at different pruning ratios beyond 0.7? Are there specific thresholds where the attack's effectiveness is significantly impacted?

From Fig 7 in our paper, we can see that when pruning ratios are over 0.7, the attack success rates drop for all EEG tasks. However, the clean accuracies drop more greatly and approach the random level. So Fine-Pruning is not that effective for Professor X.

In our understanding of Fine-Pruning, we would like to say that there is no specific thresholds where the attack's effectiveness is significantly impacted. Because the threshold in our paper seems to be 0.7, but this threshold is for EEGNet models. For other diverse EEG BCI models, many factors like the architecture, size, and activation function of the model will all have impact on the threshold.

In practical applications of Fine-Pruning, the defender will set a threshold for clean accuracy, then prune the neurons until the clean accuracy drops to the threshold. So when the threshold is set relatively high, the backdoor may not be erased.

What future research avenues could be explored to improve defenses against backdoor attacks like Professor X, particularly in the context of Fine-Pruning?

As we discussed above, Fine-Pruning requires that the defender has a validation dataset $D_{valid}$. The performance of Fine-Pruning relies heavily on the quality of the validation dataset, since the low-activated neurons are determined by the validation dataset.

So in the future, building a large, diverse, high quality, and absolutely clean validation dataset is the key for improving the Fine-Pruning's performance. The most important part is the diversity, which not only means the diversity of EEG tasks, but also means the diversity of EEG formats. Thus, improving the defenses against backdoor attacks is not an easy task and needs joint efforts of the medical and academic communities.

---

We hope our responses could help address your concerns. We believe that this work contributes to this community and has the potential to serve as a catalyst for its development. We would sincerely appreciate it if you could reconsider your rating and we are more than happy to address any further concerns you may have.

[6] Gao Y, Xu C, Wang D, et al. Strip: A defence against trojan attacks on deep neural networks[C]//Proceedings of the 35th annual computer security applications conference. 2019: 113-125.

[7] Bolun Wang, Yuanshun Yao, et al. Neural Cleanse: Identifying and mitigating backdoor attacks in neural networks. In 2019 IEEE Symposium on Security and Privacy (S&P), pp. 707–723. IEEE, 2019.

[8] Brandon Tran, Jerry Li, and Aleksander Madry. Spectral signatures in backdoor attacks. Advances in Neural Information Processing Systems (NeurIPS), 31, 2018.

[9] Kang Liu, Brendan Dolan-Gavitt, and Siddharth Garg. Fine-pruning: Defending against backdooring attacks on deep neural networks. In International Symposium on Research in Attacks, Intrusions, and Defenses, pp. 273–294. Springer, 2018.

Thank you for your thoughtful and insightful suggestions! We believe we have comprehensively addressed your questions regarding Professor X's sophisticated design, its generalizability on novel tasks, its scalability on larger or more complex datasets, and defensive research against Professor X. 

We would like to emphasize that our method is the first to achieve both highly stealthy and robust backdoor attacks on EEG BCI. Through data poisoning approach, our method even does not require controlling the training stage of target models.  

We are wondering whether you have any additional questions or comments regarding our response to your review comments. We will do our best to address them.  

We sincerely appreciate the time and effort you have dedicated to reviewing our manuscript. Thank you for your thoughtful consideration!

Dear Reviewer kEdv:

Thanks for your recognition of our work and insightful questions about how to defend Professor X. We are writing this comment to kindly request for post-rebuttal comments.

We given point-by-point answer to each of your questions and we believe we have comprehensively addressed them. Plus, we have added new experiments and a new section on the defensive research in our revised paper. Have we already addressed your concern? Or do you have any further concerns?

Please feel free to contact us, we are more than happy to hear from you. We would really appreciate if you could reconsider your rating and we are glad to address your further concerns.

Best,
Authors

Dear Reviewer kEdv:

This is a gentle reminder that the discussion period ends in about 30 hours. We are pleased to report that after engaging in the discussion, other reviewers have raised their scores: Reviewer JuBu raised 1 point and Reviewer dg1E raised 2 points. We responded to your original review on 11/19 and wanted to check in to see if you have any further questions or comments. If you find the proposed revisions and the discussion here helpful in clarifying the paper and/or increasing its value, we kindly request that you comment to that effect and consider raising your score before the deadline. Please let us know if you have any final comments, as we aim to address any of your concerns by tomorrow, 12/2. Again, the final deadline for your response is in 30 hours. Thank you for your time and thoughtful consideration!

Best,
Authors

We truly thank you for your appreciation of our work! Your questions about the generalizability of our attack is insightful, which also provides an opportunity to summarize our contributions. Our point-by-point responses are as follows.

Weaknesses

The method might encounter difficulties when scaling to larger or more intricate datasets, which could restrict its effectiveness in real-world applications with varied user populations.

We would like to clarify that our method is effective no matter how large the size of EEG datasets. As the we poison the dataset by a constant poisoning ratio, the larger the target dataset, the more poisoned data we inject into the dataset.

Or we may misunderstand the word "large", we kindly guess that does this mean the number of subjects in the dataset is large? If so, our attack is still effecitve, as the experiments performed in our paper are all under cross-subject setting. We adopt the same experiment setup as that in the paper [1]. In short, we train the EEG model on the data from n-2 subjects, and test the EEG model on the data of a new subject. Our attack works very well under this cross-subject setting, so we can conclude that our method is effective no matter how large the number of subjects in the EEG datasets.

As for the word "intricate", we kindly guess that does this mean the intricacy of classification task? Like, the number of label types. If so, we would like to argue that our attack is effective at most situations ( > 90%). Because in most cases, the number of label types in EEG BCI is no mare than four. The datasets selected in our experiment are the most intricate datasets. For emotion recognition, most datasets only focus on binary classification, like DEAP [2] and DREAMER [3]. For motor imagery, most datasets only do binary classification. As far as we know, the BCIC-IV-2a is the only dataset to do four-class classification. For epilepsy detection, most datasets only do itcal or non-itcal binary classification. We have refined the task to four categories.

From table 1, our method has been proven to be effective when facing various situations (diverse EEG models, tasks, and formats). Furthermore, we evaluate our attack on another public dataset which studies the P300 tasks [4,5]. The attack performances of three different EEG models on the dataset are still excellent:

It is worth mentioning that these results are obtained by only running the reinforcement learning 30 iterations, which takes only 0.5 hour on each model. These results can be another strong evidence to demonstrate the generalizability to other EEG datasets and real-world scenarios.

In conclusion, we are quite confident that our attack will be effective when encountering difficulties when scaling to larger or more intricate datasets.

---

[1] Meng L, Jiang X, Huang J, et al. EEG-based brain–computer interfaces are vulnerable to backdoor attacks[J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2023, 31: 2224-2234.

[2] Koelstra S, Muhl C, Soleymani M, et al. Deap: A database for emotion analysis; using physiological signals[J]. IEEE transactions on affective computing, 2011, 3(1): 18-31.

[3] Katsigiannis S, Ramzan N. DREAMER: A database for emotion recognition through EEG and ECG signals from wireless low-cost off-the-shelf devices[J]. IEEE journal of biomedical and health informatics, 2017, 22(1): 98-107.

[4] U. Hoffmann, et al. "An efficient P300-based brain-computer interface for disabled subjects", J. Neurosci.Methods, 2008.

[5] Rodrigo Ramele. P300-Dataset. https://www.kaggle.com/datasets/rramele/p300samplingdataset

Questions

What potential research directions could be pursued to enhance defenses against backdoor attacks such as Professor X, especially regarding Fine-Pruning techniques?

Thanks for your interest of defensive research against Professor X and we are also concerned about its potential dangers. We have evaluated five defensive methods to try to guard Professor X, but all of them failed. The Fine-Pruning method seems to success at the pruning ratio over 0.7, but it damages the clean task's performance.

Fine-Pruning [4] assumes that the defender has a validation dataset $D_{valid}$ in which all data are clean. The defender feeds these clean data into the backdoor models, and records the average activation of each neuron. Afterwards, the defender iteratively prunes neurons from the DNN in increasing order of average activations. In practical applications of Fine-Pruning, the defender will set a threshold for clean accuracy, then prune the neurons until the clean accuracy drops to the threshold. So when the threshold is set relatively high, the backdoor may not be erased.

As we discussed above, Fine-Pruning requires that the defender has a validation dataset $D_{valid}$. The performance of Fine-Pruning relies heavily on the quality of the validation dataset, since the low-activated neurons are determined by the validation dataset.

So in the future, building a large, diverse, high quality, and absolutely clean validation dataset is the key for improving the Fine-Pruning's performance. The most important part is the diversity, which not only means the diversity of EEG tasks, but also means the diversity of EEG formats. Thus, improving the defenses against backdoor attacks is not an easy task and needs joint efforts of the medical and academic communities.

---

We hope this response could help address your concerns. We believe that this work contributes to this community and has the potential to serve as a catalyst for its development. We are more than happy to address any further concerns you may have. Thanks again!

[4] Kang Liu, Brendan Dolan-Gavitt, and Siddharth Garg. Fine-pruning: Defending against backdooring attacks on deep neural networks. In International Symposium on Research in Attacks, Intrusions, and Defenses, pp. 273–294. Springer, 2018.

Thank you for your thoughtful and insightful suggestions! We believe we have comprehensively addressed your questions regarding Professor X's generalizability on novel tasks, its scalability on larger or more complex datasets, and defensive research against Professor X. 

We would like to emphasize that our method is the first to achieve both highly stealthy and robust backdoor attacks on EEG BCI. Through data poisoning approach, our method even does not require controlling the training stage of target models.  

We are wondering whether you have any additional questions or comments regarding our response to your review comments. We will do our best to address them.  

We sincerely appreciate the time and effort you have dedicated to reviewing our manuscript. Thank you for your thoughtful consideration!

Thanks for your valuable comments, which helps us to greatly improve our paper's quality! The additional experiments you requested have enriched our experimental results and better demonstrated the generalization and robustness of our method! We'd like to express our appreciation that our novel reinforcement learning and comprehensive experiments are well recognized. Below we carefully address your questions and concerns point-by-point.

Weaknesses

However, it would be great if the authors considered and designed some baselines based on the existing frequency-based BD attack (FreBA in the following text), if applicable.

Thanks for your careful reading! Actually we have compared our model with the vanilla frequency-based BD attack. First we would like to illustrate the difference between our attack and previous FreBA. Then your concern about the comparison of FreBA can be naturally resolved.

Compared to the previous frequency-based backdoor attack, our attack has three differences: (1) Multi-target vs Single-target: our attack can fully control the output of classifier model, but previous FreBA can only attack one target class. (2) Stealthiness of Time-Series Modality: previous FreBA are designed for image modality, which will lose stealthiness while directly performing on the time-series modality, our attack introduce a novel HF loss to address it, showing in Fig 8. (3) Reinforcement Learning: we introduce RL to optimize the injecting strategy, which greatly improves the performance, stealthiness and robustness. Previous FreBA inject the trigger in a constant place.

In the Table 3, we conducted an ablation study to verify the effectiveness of RL, please kindly refer to the revised paper. The variant Random is basically a vanilla frequency-based BD attack. The results showed our attack is better than the vanilla FreBA.

It would be great if the authors designed an experiment to validate the stealthiness of the method. It may be similar to a previous study [1], which used anomaly detection methods.

Thanks for your valuable advice! We have added this experiment in our revised paper, in Table 6. The ROC-AUC is around 0.5 and F1-score is either around 0.5 or near 0 across all datasets, indicating that the detection results are nearly random guess. These strongly demonstrates the stealthiness of Professor X.

The author only considers three models for the classifiers: EEGNet, DeepCNN, and LSTM. However, it would be great if the author considered other new models, like TimesNet [1] and other new transformer-based models.

Thanks for your constructive suggestions. We have added new experiment results in our revised paper, in Table 2. It can be seen that our attack works well (the ASRs are almost over 95%, except conformer on the ER task) on the TimesNet [1] and a new Transformer-based model EEG-conformer [2] (is cited over 200 times), further proving the generalizability of our attack. We have detailed the individual ASRs for each category in Table 1 of the main text, please kindly refer to the revised paper.

It takes a lot of time to conduct the experiment with TimesNet, as this model is not designed for multi-channel data like EEG BCI, thus is inefficient in processing EEG signals. We use TimesNet as a singal-channel feature extrator and concatenate all features of all EEG channels. Then we feed the concatenated feature into a linear layer for EEG classification. Our code of TimesNet and EEG-conformer will be public too.

By the way, we would like to discuss the reasons why we chose EEGNet, DeepCNN and LSTM. These three models are the most widely used model in the EEG community. And for most real-world application of EEG BCI, only shallow and simple models are required because deep and sophisticated models may overfit when the data is limited. Thus, considering the real-world situation, we chose these three simple models, but it doesn't mean that our attack is not effective on new sophisticated models.

We would like to take this opportunity to emphasize that our method is an EEG task-agnostic, model architecture-agnostic, and format-agnostic attack, which can generalize to many scenarios.

---

[1] Wu H, Hu T, Liu Y, Zhou H, Wang J, Long M. TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis. In The Eleventh International Conference on Learning Representations.

[2] Yonghao Song, Qingqing Zheng, Bingchuan Liu, and Xiaorong Gao. EEG conformer: Convolutional transformer for EEG decoding and visualization. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 31:710–719, 2022.

The quality of the writing needs improvement; here are some points: The third paragraph of the introduction requires revision for clarity and coherence. Figure 1 consists of five sub-figures that provide a good summary of the method. However, in the introduction (line 050), the authors begin by explaining Figure 1-d, which destroys the flow.

Your suggestions are so helpful, thank you very much! We have reorder the subfigure in Figure 1 and refine our writing in the revised paper. Please kindly refer to the new version of our paper, where the modifications are marked in blue.

The Methodology section should be improved by first defining the key concepts, symbols, and problems. It would also be helpful to include a table of abbreviations and symbols, as the multiple terms used throughout the paper may be confusing for readers.

What a great suggestion! We added a table of key symbols in the Appendix A (Table 7) due to the page limitation. And we remind the reader the symbol table at the beginning of methodology section. Please kindly refer to the new version of our paper.

Questions

How effective is the proposed BD attack when applied to approaches that utilize frequency information of data, such as [3]?

We are very sure that our attack is effective on any supervised learning-based models no matter they utilize frequency information or not. Because the TimesNet model contains a module to convert 1D time series data into structured 2D tensors using the frequency information, and our attack works well on TimesNet. So our attack is effective when applied to approaches that utilize frequency information of data.

However, the paper [3] is a self-supervised contrastive learning method, which is not supervised learning. Even in the image field, BD attacking self-supervised learning is far different from BD attacking supervised learning [4], not to mention BD attacking contrastive learning [5]. Thus, we don't know whether our attack can work on [3], but we would like to kindly remind that the effectiveness of BD attack on [3] is out of our paper's scope.

---

We hope our responses could help address your concerns. We believe that this work contributes to this community and has the potential to serve as a catalyst for its development. We would sincerely appreciate it if you could reconsider your rating and we are more than happy to address any further concerns you may have. Thanks again!

[3] Zhang X, Zhao Z, Tsiligkaridis T, Zitnik M. Self-supervised contrastive pre-training for time series via time-frequency consistency. Advances in Neural Information Processing Systems. 2022 Dec 6;35:3988-4003.

[4] Jia J, Liu Y, Gong N Z. Badencoder: Backdoor attacks to pre-trained encoders in self-supervised learning[C]//2022 IEEE Symposium on Security and Privacy (S&P). IEEE, 2022: 2043-2059.

[5] Carlini N, Terzis A. Poisoning and Backdooring Contrastive Learning[C]//International Conference on Learning Representations. 2022, oral.

Thank you for your thoughtful and insightful suggestions! We believe we have comprehensively addressed your questions regarding Professor X's comparison with vanilla FreBA, its stealthiness against anomaly detection methods, its generalizability on new sophisticated models, and writing improvement. We added the additional experimental results and refined our writing in our revised submission. 

We would like to emphasize that our method is the first to achieve both highly stealthy and robust backdoor attacks on EEG BCI. Through data poisoning approach, our method even does not require controlling the training stage of target models. 

We are wondering whether you have any additional questions or comments regarding our response to your review comments. We will do our best to address them. 

We sincerely appreciate the time and effort you have dedicated to reviewing our manuscript. Thank you for your thoughtful consideration!

We thank the reviewer for your time and effort in reviewing our work. Below we would like to exchange our ideas with you point-by-point.

It remains inaccurate to assert that "While electroencephalogram (EEG) based brain-computer interfaces (BCIs) have been extensively employed in medical diagnosis, healthcare, and device control,..."

We would like to say that EEG BCIs have been widely employed is not wrong. Because the employment in academia is also employment. As you agree with that BCI research is experiencing significant growth due to advancements in machine learning. According to the Web of Science, the number of papers on EEG BCI has increased from 6,185 in 2010 to 16,470 in 2023.

Our attack can be used for falsifying EEG datasets, which is significantly severe in academia. As one can draw any neuroscience conclusion from a fake EEG dataset. The negative impact of academic fraud goes without saying, especially in the field of life sciences.

The submission relies on historical datasets such as BCIC-IV-2a. Moreover, the methodology seems to rely on a dubious SEED database.

We adopt three public dataset with high citation to ensure our experiments are credible. These datasets are covering three widely-studied EEG tasks: 1) epilepsy detection (CHB-MIT), 2) emotion recognition (SEED), 3) and motor-imagery (BCIC-IV-2a).

Since you questioned the BCIC-IV-2a and SEED dataset, we would like to know your opinion about the epilepsy detection CHB-MIT dataset. What do you think about the CHB-MIT dataset? Our attack is also effective on the CHB-MIT dataset, is this result unrealistic too?

We sincerely hope that you could explain why you evaluate the SEED dataset "dubious", and what factors do you refer to when you say "dubious"? According to the SEED Dataset's Website [1]: As of December 2023, the cumulative number of applications and research institutions using SEED have reached more than 5800 and 1000, respectively. Moreover, The SEED's paper has been cited by more than 1,900 papers. Are these 1900 papers dubious too?

It is critical to note that motor imagery may not be effective for the target demographic of individuals with paralysis due to significant neural degeneration.

We would like to say that this fact doesn't mean the useless of motor imagery EEG BCI. According to your logic, we can say that It is critical to note that GLASSES may not be effective for the target demographic of individuals with MYOPIA due to significant neural degeneration. However, for other people with myopia due to other reasons such as long screen time, the glasses is very helpful. Hence, the fact you presented has nothing to do with the effectiveness of motor imagery EEG BCI.

In contrast, for the many patients with paralysis due to other reasons like spinal cord injuries, the motor imagery BCI is very helpful. The EEG BCI already helps a paralytic walk again [2], so the wider application of EEG BCI is predictable.

To be honest, we did not understand the rationality of your logic, why you present this fact and what do you want to illustrate by this fact? We sincerely hope you could explain it.

Questions

Why did the authors construct an entirely unrealistic and artificial scenario?

No, we didn't. Please kindly refer to the general response for more details.

Where did the authors encounter such hyperbolic or enthusiastic claims regarding the purported applications of BCI in healthcare and medical diagnostics?

We, as EEG BCI researchers, are conducting cutting-edge EEG BCI research with multiple top hospitals to try to improve the life quality of epilepsy patients. Specifically, we are trying to provide early warning for epileptic seizures with the help of EEG BCI. We are quite sure that the research of epilepsy detection is meaningful.

As the impressive results presented in [2], where a paralytic walk again with the help of EEG BCI, the motor imagery EEG BCI has been the hope of many paralytics.

Currently, only a limited number of conditionally approved, mostly invasive devices have been tested on a small cohort of subjects within closed clinical studies.

1. We would like to argue that we don't have to wait until the horse has bolted before close the stable door.

2. Although there are a limited number of subjects are taking the treat of BCI, we would like to say we still need to consider the security risk of the using of EEG BCI on them.

3. Please note that our attack can also be misused in academic researches for academic fraud.

---

We hope this response could help address your concerns. We believe that this work contributes to this community and has the potential to serve as a catalyst for its development. We are more than happy to further exchange ideas with you.

[1] https://bcmi.sjtu.edu.cn/home/seed

[2] Lorach H, Galvez A, Spagnolo V, et al. Walking naturally after spinal cord injury using a brain–spine interface[J]. Nature, 2023, 618(7963): 126-133.

Thank you for your thoughtful and insightful suggestions. We believe we have comprehensively addressed your questions regarding Professor X's realism.

We are wondering whether you have any additional questions or comments regarding our response to your review comments. We will do our best to address them. 

We sincerely appreciate the time and effort you have dedicated to reviewing our manuscript. Thank you for your thoughtful consideration!

Dear Review RejD:

Thank you for your feedback and the valuable time in reviewing our paper! We would really appreciate if you can reply some of our response's questions. Since your latest feedback (not public to everyone and ACs) overlooked these questions. We list them below for saving your valuable time:

1. Why do you regard the SEED dataset dubious?
2. What's your opinion about the CHB-MIT dataset, are our attack's results on this dataset unrealistic too?
3. What do you want to illustrate by this fact "It is critical to note that motor imagery may not be effective for the target demographic of individuals with paralysis due to significant neural degeneration"?

We sincerely hope to hear from you and discuss with these questions. This is important for us to better address your concerns.

Best,
Authors

A. Why do you regard the SEED dataset dubious?

Answer: The SEED database, while popular, has methodological limitations. The long-duration film clips, despite single-emotion labeling, likely induced multiple affective states, compromising the accuracy of self-reported emotions. The laboratory setting may not fully capture natural emotional expression, and the focus on basic emotions limits its applicability to complex emotional states. Cultural biases, inherent in using a limited dataset, further complicate the interpretation of results. Additionally, the use of extended film clips to induce specific emotions for BCI applications may not be practical or effective.

B. What's your opinion about the CHB-MIT dataset, are our attack's results on this dataset unrealistic too?

Answer: The proposed scenario of a malicious actor remotely hacking a medical device to induce seizures in patients is highly unrealistic and impractical. Medical devices, especially those involved in life-critical functions, are typically isolated from public networks for security and regulatory compliance reasons. Cyberattacks on such devices almost always require physical access or sophisticated social engineering techniques. A more realistic and relevant research direction would be to focus on improving the security of medical device networks and developing robust defense mechanisms against potential threats. This could involve research on advanced firewall technologies, intrusion detection systems, and secure firmware updates.

C. What do you want to illustrate by this fact "It is critical to note that motor imagery may not be effective for the target demographic of individuals with paralysis due to significant neural degeneration"?

Answer: Sensorimotor Rhythm (SMR)-based motor imagery BCI is widely recognized as unsuitable for late-stage ALS (CLIS) patients due to cognitive decline and altered brain activity patterns [1,2]. While effective in earlier stages, when numerous non-BCI alternatives exist, alternative BCI approaches like P300-based or fNIRS-based systems are more appropriate for CLIS patients. While recent research has questioned certain aspects of [2], this seminal work remains a cornerstone in the BCI field. Therefore, exploring attacks on a paradigm with limited practical application in late-stage ALS (CLIS) is not an appropriate submission for a top-tier conference such as ICLR 2025.

1. Foerster BR, Welsh RC, Feldman EL. 25 years of neuroimaging in amyotrophic lateral sclerosis. Nat Rev Neurol. 2013 Sep;9(9):513-24. doi: 10.1038/nrneurol.2013.153. Epub 2013 Aug 6. PMID: 23917850; PMCID: PMC4182931.

2. Kübler A, Birbaumer N. Brain–computer interfaces and communication in paralysis: Extinction of goal directed thinking in completely paralysed patients?. Clinical neurophysiology. 2008 Nov 1;119(11):2658-66.

We greatly appreciate your timely response and valuable comments on the datasets! We are now more likely to know what your real concern is, and would like to kindly conclude it as: The paradigm and applications of EEG BCI are not mature enough at the current stage, thus there is no need to focus on the safety issue.

Major point

We would like to take this opportunity to emphasize the motivation of our paper, which will in some degree answer your concern:

1. Although the BCI applications is not mature enough at the current stage, it is still necessary to pay attention to backdoors like Professor X for future BCI applications. We don't have to wait until the horse has bolted before close the stable door. It is worth noting that EEG BCI is gradually moving towards practical application [1]. We are confident that BCI will have better practical applications in the future.

2. Our work provides a fundamental security assessment perspective for BCI society, promoting the real practical application of BCI. Since the primary consideration before real application is security issues.

3. EEG BCI is widely used in academia, our attack can be used for falsifying EEG datasets. We provided an example in the General Response to illustrate the harm of this malicious use, which we absolutely do not encourage. We hope that our research can serve as a catalyst in the BCI community, raising more attention to BCI safety and promoting its practical application.

Minor point

By the way, as you said P300-based method is more effective for CLIS patients, we also validated our attack on a P300 dataset [2] and found our attack is still effective. But after all the CLIS patients do not represent everyone who needs a motor imagery BCI, there are many other patients like those suffering from spinal cord injuries need motor imagery BCI. We admit motor imagery BCI is not effective for everybody, but as long as one patient benefits from it [1], it is useful.

---

In conclusion, our work is not aiming to attack BCI that has already been applied in the real-world (while Professor X has the ability), but more about alerting a severe potential hazard of backdoor in EEG BCI and calling for defensive research (that's why we added a new section in appendix to discuss how to defend backdoor like Professor X in EEG BCI).

The datasets used in our study is for validating our attack's efficacy across various EEG tasks and formats. There might be some drawbacks in these datasets, but after all it is not the focus of our work. Our work promotes the future practical application of EEG BCI (in the safety way) and the construction of a more honest academic community (in the academic integrity way).

Thanks for your valuable time in discussing with us! We are more than happy to address any further concerns you may have.

[1] Lorach H, Galvez A, Spagnolo V, et al. Walking naturally after spinal cord injury using a brain–spine interface[J]. Nature, 2023, 618(7963): 126-133.
[2] Rodrigo Ramele. P300-Dataset. https://www.kaggle.com/datasets/rramele/p300samplingdataset

Dear Reviewer RejD:

Thanks for your timely and detailed response! We are really glad that we and you have already reached some consensus. Below we tried to address your concerns:

Unrealistic problem.

1. We definitely agree with that the EEG BCI is not mature at the current stage, including the paradigm and applications. But the potential practical application of it cannot be denied (We believe you are agree with this as you are a member of the BCI community too). Before real application of EEG BCI, we must consider the security issues of it. Our work promotes developing more safer BCI for future real application, which is definitely not unrealistic.

it's crucial to recognize that future approved medical devices are subject to stringent regulations and security measures, limiting the potential for malicious exploitation.

2. We believe we and you have reached the consensus that the primary consideration before real application is security issues. Unfortunately, the security issues are definitely not limited by simple and vague "stringent regulations and security measures". First of all, how to draft these regulations? If all members in the BCI community do not realize that BCI can be injected backdoor, there is no one to draft the regulations to regulate backdoor attacks in BCI. Our work provides an important insight about the severe backdoor attacks threats to EEG BCI. However, we believe that there must be other security issues in BCI but are never been noticed. It would be dangerous if someone maliciously used some techniques unknown to the public. Our work promotes the draft of regulations for Backdoor Attack in BCI.

The medical device field, particularly regarding pacemakers, DBS, ... This maturity has led to robust security protocols and regulatory oversight, such as those enforced by the FDA, EMA, etc.

3. The medical devices you cited including pacemakers, DBS, cochlear implants, etc, are not using the AI or deep learning (DL). So they definitely have not been affected by backdoor attacks aiming at AI models. These devices are developed based on the traditional electronic methods and the industry has adequate knowledge about these devices along with the potential hazard they may face, that's why robust security protocols and regulatory oversight can be made by the FDA, EMA, etc.
   We hope to reached the consensus with you that AI or DL has shown some superiority in BCI, due to the complexity and human-unreadbility of brain signals. In recent years, there has been an increasing number of articles developing EEG BCI with AI or DL methods. However, the BCI community dose not have sufficient knowledge about AI or DL models along with the potential hazard they may face. Our work is aiming to study and point out the potential hazard, promoting designing more safer BCI for future real application.

User model customization for each session, necessitated by electrode placement variations, remains a significant challenge in EEG-based BCI. This practical issue warrants further research attention rather than theoretical explorations of unrealistic backdoor attacks... We urge you to consider applying your research efforts to fields such as image processing or speech recognition...

4. Actually, we are BCI researchers, not safety researchers in the image processing or speech recognition fields. We definitely agree with that the studies for user model customization and electrode placement, etc, are essential in BCI. However, studying security issues is equally important, and these two are not in conflict. Just like the traditional mature medical devices, there are some researchers focus on improving their performance, while there must be some researchers focus on improving their safety.
   We noticed the security issues of EEG BCI, although we know the whole BCI society is focusing on improving the decoding accuracy and robustness, we can not ignore the security issues and pretend that we didn't notice. Our work firstly prove that EEG BCI can be injected with invisible and robust backdoor, and unfortunately the backdoor can not be detected by any existing methods. Our work provides valuable insight about the security issues of EEG BCI and promotes the draft of regulations for limiting Backdoor Attack in BCI.

Potential hazard in academia

5. Let's set apart the practical application of EEG BCI and talk about the academia. As we all know, EEG BCI is not only a medical devices, but also a good neuroimaging techniques because of its portability and low cost, which is widely used in the neuroscience society. Our attack can also be misused for falsifying EEG datasets and drawing completely wrong neuroscience conclusion as we discussed in the General Response. Our work promotes the construction of a more honest neuroscience society as we pointed out this cheating methods.
   We think this cheating is probably the most severe hazard that can work now.

Best,
Authors