Threaten Spiking Neural Networks through Combining Rate and Temporal Information

To Reviewer 4VCV

Thanks for your invaluable and constructive feedback! We are encouraged that you find our paper novel and relevant to the community, as well as outperforming related works. We would like to address your concerns and your questions in the following.

Q1: In Table 1, please clarify what results are relative to CBA and what results are relative to the proposed attack.

A1: Thanks for pointing it out! In Table 1, the regular font on the left side of the table represents CBA (the baseline method), while the bold font on the right side represents our proposed attack method. We have made a clarification in our newly submitted version.

Q2: Please provide more details of the intuitions and design decisions made to develop Eq.12.

A2: From Figure 1, it is evident that the same average input current ($W^lr^{l-1}(T)$) can lead to different spike firing rates ($r^l(T)$) due to the non-uniform distribution of input current across multiple consecutive time-steps. Previous studies on ANN-SNN conversion [1, 2] have found that assuming a uniform input current, we have the relationship $r^l(T) = \frac{1}{T}\text{clip}\left( \left\lfloor \frac{TW^lr^{l-1}(T)+v^l(0)}{\theta^l} \right\rfloor, 0, T \right)$. This implies that the average firing rate of adjacent layers follows an approximate linear transformation, and there exists a one-to-one mapping between $r^l(T)$ and $W^lr^{l-1}(T)$. Although the equation $r^l(T) = \frac{1}{T}\text{clip}\left( \left\lfloor \frac{TW^lr^{l-1}(T)+v^l(0)}{\theta^l} \right\rfloor, 0, T \right)$ is based on the IF model, it holds true that the one-to-one mapping relationship still applies to general LIF models. In this scenario, the SNN is predominantly influenced by rate information, and the computational relationship between two adjacent SNN layers is entirely equivalent to that of an ANN. Moreover, in this situation, for any $W^lr^{l-1}(T)$, the variance of $\frac{r^l(T)}{W^lr^{l-1}(T)}$ is zero, that is, $\mathbf{Var}\left( \frac{r^l(T)}{W^lr^{l-1}(T)} \right)=0$.

For SNNs with more temporal information, the same average input current ($W^lr^{l-1}(T)$) can lead to different spike firing rates ($r^l(T)$), and the variance of $\frac{r^l(T)}{W^lr^{l-1}(T)}$ is not zero. Thus, it is reasonable to consider to use the variance of $\frac{r^l(T)}{W^lr^{l-1}(T)}$ to measure the temporal information and we propose Eq. 12 to measures the richness of temporal information according to the expectation of the variance. As $\chi^l$ increases, the variations in firing more or fewer spikes due to different spike arrival sequences in Figure 1 become more pronounced. This signifies a stronger influence of temporal information within the SNN.

Q3: Please describe the proposed attack through a detailed algorithm that collects all the operations and equations involved.

A3: Thanks for your constructive suggestion! We have made a detailed algorithm to describe our HART attack framework step by step. Please refer to Section A.3 in the Appendix of our newly submitted version.

To Reviewer w1gJ

Thanks for your insightful and constructive feedback! We are encouraged that you find our paper novel, solid, well-written and highly noteworthy, as well as providing a rigorous theoretical analysis. We would like to address your concerns and your questions in the following.

Q1: I find it hard to derive equation 8 by combining equations 1 and 2. I am wondering about the absence of the threshold in equation 8. The authors should clarify why the threshold is not included in equation 8.

A1: Thanks for pointing it out. To simplify the subsequent mathematical analysis, we actually set $\theta^l=1$ here. We have clarified in page 3 and added detailed derivation in the Appendix, Section A.6.

Q2: How about defense? Can the proposed method also be used to improve the robustness of SNNs.

A2: Thanks for your constructive comment! We have explored the use of images perturbed by the HART gradient as adversarial samples to enhance the robustness of SNNs through adversarial training. Our experiments encompass both vanilla SNNs and robust SNNs obtained via HART adversarial training, evaluated under FGSM and PGD attack scenarios with a white-box setting. We employ the attack success rate as our metric and set $\lambda$ to 0.9. As depicted in Table R1, our results demonstrate a significant improvement in the defense capability of robust SNNs across various attack scenarios, including BPTR, STBP, and RGA attacks. Notably, during our adversarial training, we did not introduce adversarial samples specifically tailored to these three mentioned attack algorithms. This finding indicates that HART not only enhances the defense capability of SNNs but also exhibits superior generalization ability, making it applicable for defending against a wide range of attack algorithms.

Table R1: Comparison between the vanilla and robust models on the CIFAR-100 dataset.

$*$ denotes a stronger PGD attack ($\alpha=2.55,\text{steps}=7$).

Q3: Please provide a more comprehensive explanation of Fig.2$\text{(c)}$.

A3: Thank you for your insightful comment! We have made improvements to Fig. 2$\text{(c)}$ in our newly submitted version. In the updated figure, we illustrate the impact of different scales of $\gamma$ on the gradient discrepancy. We specifically select three spiking neurons with distinct $m^l(t)$ values at the $t$-th time-step. It can be observed that as $\gamma$ decreases, the discrepancy of surrogate gradients among the spiking neurons increases. Consequently, HART tends to focus on a subset of spiking neurons that fall within a specific membrane potential range at each time-step. This leads to the gradient with enhanced temporal attributes. Conversely, as $\gamma$ increases, the gradient discrepancy among the three spiking neurons diminishes, and HART exhibits a more pronounced rate attribute.

To Reviewer RoZ6

Thanks for your constructive and thoughtful comments! We are encouraged that you find our paper interesting and effective, as well as providing detailed experiments. We would like to address your concerns and your questions in the following.

Q1: It is better to provide an illustration of how to utilize temporal information during BP (as Fig.1).

A1: Thanks for your advice. We have improved Figure 2 and the related content to better illustrate the details of utilizing temporal information during BP in our newly submitted version.

Q2: Does the pruning has negative effect on the gradient computation? Whether the proposed gradient computation can be directly applied to SNN training?

A2: Thanks for your insightful comment! We would like to clarify that in this work, the gradient pruning and merging techniques are combined within the HART framework to enhance the extraction of rate and temporal information. This combination has a positive impact on improving attack success rates. In Table A1 (found in the Appendix), we present a series of ablation experiments that compare "STBP+optimal $\gamma$" with HART. The key distinction between the two lies in the inclusion or exclusion of pruning and merging. The experimental results demonstrate that pruning and merging indeed contribute to improved attack effectiveness. Thanks for your suggestion! We agree that the potential application of these gradient techniques to SNN training is a fascinating direction for further research. We have included this point in the conclusion section.

Q3: It is better to discuss the complexity of gradient computation in HART.

A3: Thanks for pointing it out. As discussed in the "Pre-calculation Property" section, we employ a pre-calculation technique for the spiking neuron layers in the HART framework. This involves calculating the sum of surrogate gradients across multiple time-steps. By doing so, we can compute the gradient of HART in a single step during the back-propagation process, similar to ANNs. Consequently, the time complexity of pre-calculation for spiking neurons is $O(T)$, while the gradient computation complexity for each synaptic layer remains at $O(1)$. In the case of deeper SNNs with more synaptic layers, we believe that this property will lead to a more significant reduction in computational cost compared to the vanilla STBP attack. We have also added the discussion in the Appendix, Section A.4.

Q4: In Fig.2(b), it is not clear the meaning of solid arrows. Fig.2$\text{(c)}$ is hard to understand.

A4: Thanks for pointing it out. We have made improvements to Fig. 2(b) and Fig.2$\text{(c)}$ in the revised version. In the updated figures, the solid lines representing gradient pruning and merging are connected to Fig. 2(a), indicating that the design of these techniques ensures the establishment of the pre-calculation property in HART during forward propagation. The solid line related to the adjustable surrogate gradient is connected to Fig.2$\text{(c)}$. In Fig.2$\text{(c)}$, we illustrate the impact of different scales of $\gamma$ on the gradient discrepancy. We have selected three spiking neurons with varied $m^l(t)$ values at the $t$-th time-step. It can be observed that as $\gamma$ decreases, the discrepancy of surrogate gradients among spiking neurons increases. This causes HART to focus on a subset of spiking neurons within a specific membrane potential range at each time-step. Consequently, this enables us to observe a gradient with more pronounced temporal attributes.

To Reviewer hCfa

Thanks for your insightful and invaluable comments! We are encouraged that you find our paper novel and solid, as well as taking an important initial stride, having theoretical derivations and exhibiting remarkable performance improvements. We would like to address your concerns and your questions in the following.

Q1: The authors should provide a more detailed analysis on the specific role of gradient pruning and merging.

A1: Thanks for your constructive comment! Gradient pruning and merging have two primary functions. Firstly, they facilitate the comprehensive utilization of both rate and temporal information. Pruning prevents gradient interference between different time-steps, while merging performs overall statistics for surrogate gradients, specifically the membrane potential information, across multiple time-steps. The synergistic use of pruning and merging promotes the integration of rate and temporal information, thereby guaranteeing the establishment of Theorem 2. Secondly, gradient pruning and merging ensure the desirable pre-calculation property of the HART framework, which requires only a single-time gradient calculation for each synaptic layer, effectively reducing the computational complexity of the algorithm.

Q2: The figures and legends of the paper can be further improved.

A2: Thanks for your advice! We have made modifications for the figures and legends in this paper, and submitted a new version.

Q3: In Fig.1, it is unclear why the purple curve appears to be identical to the curve representing case 1.

A3: In fact, there is no correlation between these two curves. We have revised the colors of Fig.1 in our newly submitted version.

Q4: I kindly request the authors to assess which specific information (rate or temporal) was leveraged by CBA, STBP, BPTR, and RGA.

A4: CBA directly transfers the gradients obtained from pre-trained ANNs to SNNs by leveraging the rate information, assuming a uniform input current. STBP utilizes gradients calculated through spatial-temporal back-propagation across multiple consecutive time steps, primarily relying on temporal information. BTPR and RGA employ the vanilla forward propagation of spiking neurons while incorporating rate information during gradient calculation. BPTR masks the gradient based on whether a neuron has emitted spikes, while RGA replaces the surrogate gradient function used in STBP with the activation gradient obtained from the Rate-Input Curve.

Q5: Given that CBA also employs rate information for its attack strategy, could the authors elucidate the factors that make the proposed rate gradient method superior?

A5: Thanks for pointing it out. In terms of forward propagation, CBA relies on the assumption of uniform input current to determine the output in each layer, whereas our approach utilizes the actual output of spiking neurons across multiple time steps. When it comes to gradient calculation, CBA typically employs a conventional activation function like ClipReLU, whereas we derive our activation gradient by averaging the statistical information of spike firing on a layer-by-layer basis. We believe that these two aspects have enabled our method to achieve more precise gradient estimations.