Temporal Misalignment in ANN-SNN Conversion and its Mitigation via Probabilistic Spiking Neurons

We thank the reviewer for their time in assessing our paper and their constructive comments.

3. We consider the setting from Theorem 1: we assume that the accumulated membrane potential in the first phase is $v:=TX$, that the threshold $\theta=1$ and that $0\leq TX\leq T$ (other cases being trivial). Let us also put $S[t]=\sum_{i=1}^ts[i]$. We consider two different situations:

Case 1. $TX$ is an integer. In particular, Theorem 1 (b) tells us that at each time step $t$, as long as $v\geq S[t-1]$, the expectation of having a spike at $t$ given $S[t-1]$ is $\frac{v-S[t-1]}{T-t+1}$. We note that the last quantity is never strictly bigger than 1 and never strictly below 0. In particular, the conditional expectation (of mem. potential before spiking) $E[v[t+1]\mid S[t-1]]=v-S[t-1]-\frac{v-S[t-1]}{T-t+1}=v[t]+\frac{v[t]}{T-t+1}$. The total expectation then becomes $E[v[t+1]]=E[v[t]]-\frac{E[v[t]]}{T-t+1}=E[v[t]]\frac{T-t}{T-t+1}$. Initially, we have $E[v[1]]=v=TX$. Solving this recurrence gives $E[v[t+1]]=v\prod_{i=1}^t\frac{T-i}{T-i+1}=v\frac{T-t+1}{T}$. Hence, $E[s[t]]=\frac{E[v[t]]}{T-t+1}=\frac{v}{T}$.

In particular, since the total number of spikes during $T$ steps is $v$ (Theorem 1. (c)) it follows that when $v$ is an integer in $[0,T]$, $S[t]$ is a random variable that follows hypergeometric distribution $H(T,v,t)$. The variance of the firing rate is then $V[\frac{1}{t}S[t]]=\frac{v}{tT}(1-\frac{v}{T})\cdot\frac{T-t}{T-1}$. Note that at the final step $T$, the firing rate is precisely $X$ with variance 0.

Case 2. When $TX$ is not an integer, we no longer have a nice expression for the probability of having a spike at $t$. Namely, the modified expression becomes $E[s[t]]=\frac{\min(\max(v[t],0),1)}{T-t+1}$ as $v[t]$ in this case can be strictly negative or be strictly larger than 1. Hence, we cannot solve the resulting recurrence in a closed form. However, we do note that the final number of spikes at step $T$ is either $\lfloor TX\rfloor$ or $\lceil TX\rceil$ (Theorem 1. (c)). Both cases occur with non-zero probabilities $p$ and $1-p$n Then, the firing rate at the end is either $\frac{\lfloor TX\rfloor}{T}$ with probability $p$, either $\frac{\lceil TX\rceil}{T}$ with probability $1-p$. The expected firing rate and its variance are $\frac{\lfloor TX\rfloor}{T}+\frac{1-p}{T}$ and $\frac{p(1-p)}{T^2}$. In particular, we see that the firing rate stabilizes with $T$.

It seems that our spiking process offers a continuous extension of hypergeometric distribution (with continuous $v$), but we were not able to precisely detect this distribution in the existing literature. Further theoretical study of this process seems like a promising continuation of this work.

To shed more light on the situation, we provide the following plots. In the first plot, we fix $T=16$ and vary $v$ continuously. We plot probability (blue) and variance (orange) of the event $S[T]=i$, when $v$ varies through $[i,i+1]$.

Prob var

We also plot the probabilities for various steps $t$ of having a spike at that time step, in dependence of the inital $v$. We notice that when $v$ is an integer, the probability is uniform for all $t$, while in other cases it varies from step to step.

Prob spike

2. We thank for the references provided. Comparison with [1] and [2]: We focus on Section 5.2 in [2] and in particular Theorem 3. We note the assumption that the assumption of the theorem is that the firing mechanism of spiking neurons are stationary processes with a fixed given number of produced spikes. However, in our case, as we discussed above, our proposed firing mechanism is dynamic, with bias depending on the emitted spikes. Furthermore, the number of spikes TPP neuron produces is ``guaranteed'' ($\lfloor TX\rfloor$ or $\lfloor TX\rfloor+1$) which offers more control in conversion. Based on our ablation studies (please refer to tables provide to Reviewer BQ8A), TPP neurons constantly outperform probabilistic neurons with fixed bias, and we expect that implementation of TPP with method of [2] would further ameliorate their respective results. We will deal with this more thoroughly in the future.

1. You are on the point in your description of the motivation behind this work and the shuffling effect (See also Theorem 1 in Appendix G). However, we do emphasize subtle characteristics of our proposed Bernoulli firing process.

4. Our work in progress deals with extension of TPP neurons to approximate non-linear (activation) functions. Please refer to Going beyond convolution architectures answer to the Reviewer UeYr.

5. We acknowledge that the stochastic component may induce extra energy consumption for TPP neurons. However, we argue that low number of SOPs to achieve high accuracy may compensate for this energy consumption.

Other Strenths and Weaknesses

(a) Theoretical justification:

We argue that ``temporal misalignment'' happens primarily due to the fact that in ANN-SNN conversion, SNN models will need a few time steps to accumulate enough potential to start firing.

We start by noting that in ANN-SNN conversion, at a particular activation layer of ANN, the thresholds for the corresponding spiking neurons SNN layer are chosen based on the distribution of the corresponding ANN activations, standard choice being the maximum activation or its percentile. In any case, there will be substantial activations that fall below this threshold. We argue that TPP neuron will produce more spikes at a first time step, than a vanilla spiking neuron, because of its probabilistic nature. To make this formal, we propose the following:

Proposition Consider an ANN neuron with ReLu activation and let activation values follow distribution with PDF $p(x)$. Let further $\theta>0$ be the threshold of the corresponding vanilla SNN neuron. Then:

• The expectation of a spike output at step $t=1$ of a vanilla spiking neuron is $\int_{\theta}^\infty p(x)dx$.

• The expectation of having a spike at step $t=1$ of a TPP neuron is $\int_{0}^\infty p(x)dx$ which is the same as the expectation of the output of the ANN neuron with ReLU activation.

Many of the baselines propose to start with an initial membrane potential, in order to ``encourage'' early spiking, however, there will always be a substantial part of the values that will fall below the threshold and the above proposition still holds.

Empirical evidence for this is presented in Figures 4 and 7, to which we kindly refer. This absence of spikes in the first time steps further causes unevenness error of the spike outputs, as thoroughly discussed, for example, in AAAI’23, “Reducing ANN-SNN Conversion Error through Residual Membrane Potential.” Ideally, we expect the spikes received from the preceding layer to be uniformly distributed.

In particular, this result shows that TPP neurons with their designed probabilistic spiking approximate the output of the ANN neurons as starting with the first time step, while for vanilla spiking neurons this approximation is coarser. Our Theorem 1 (b) then shows how this approximation evolves throughout the rest of the simulation time.

Other Comments Or Suggestions

Going beyond convolution architectures In our current work in progress, we consider generalization of TPP neurons that are specific to approximate various non-linear activation functions, as a first step towards conversion of various ANN architectures. In particular, design that we are currently exploring is the following (please see the following Figure for a visual explanation).

activation function

We consider a general nice enough activation function $f$, and a sequence of thresholds $\theta_1,\dots,\theta_T$, chosen in such a way that the steps with length $\theta_i$ and height $s=1$ approximate the function $f$ in an ``optimal'' way (we do not discuss how to choose $\theta_i$ and what optimality would actually mean). If we denote by $\Theta_t:=\sum_{i=1}^t\theta_i$, the modified spiking mechanism of our TPP neuron becomes $s[t] = B(\frac{v[t-1]}{\Theta_{T-t+1}})$, $v[t+1] = v[t]-s[t]\theta_t$. We note that the mechanism generalizes our TPP in that for ReLU activation we had $\theta_t=\theta$ for all $t$. This situation requires more sophisticated approach for theoretical insights, and we decided to keep it separate from our submission.

Questions For Authors

For the experiments in the main body of the paper, we performed permutations with 5 different seeds and report the mean of the results. However, we kindly refer to Appendix G where we provide further experiments concerning permutations as well as theoretical insights into their functioning (Appendix G, Theorem 1). We would also like to point one curiosity here, as it is still beyond our understanding. Namely, in Figure 11, we report results when for latency $T=4$ and baseline model, a chosen permutation is applied to all the spiking layers of the model (so the permutation does not vary from layer to layer in a random way). We tested for all the 24 possible permutations of 4 elements and for all the 24 permutations, the performance improved.

Supplementary material

We provide anonymized code that was used in some of the experiments.

Code

We thank the reviewer for their time and constructive comments.

Claims And Evidence: Not Clear

It is important to ensure not only that the expected number of spikes matches the value of the ANN activation, but also that these spikes are distributed in a uniform manner. In particular, even though all spiking neurons in one layer emitted the expected number of spikes, when we apply the subsequent weights the next layer will not necessarily receive the expected input corresponding to the input of ANN layer. The key issue here is the unevenness error, as thoroughly discussed in AAAI’23, “Reducing ANN-SNN Conversion Error through Residual Membrane Potential”, as it can happen that the expected number of spikes arrives too early or too late.

So, ideally, we expect the spikes received from the preceding layer to be uniformly distributed. However, as spikes propagate through deeper layers, their timing often becomes increasingly irregular, leading to deviations from the expected spike count and resulting in unevenness error.

For your convenience, we compared our TPP method with a naive probabilistic spiking neuron, where after the accumulation of the membrane potential $v$, the neuron emits spikes with the constant bias $v/(\theta\cdot T)$ (so spiking becomes a stationary process). The following tables show that our design is outperforming the stationary process. Our Theorem 1. (b) offers an explanation, as the dynamic bias takes into account already emitted spikes and offers a more meaningful approximation of the ANN outputs. We will provide full tables in the revised manuscript.

Table1 and Table2

Experimental Designs Or Analyses

The shuffling is performed in the following way. Once a baseline method is fixed, we proceed in order, layer by layer. After each spiking layer we would collect spike trains of that layer, then we would permute them in temporal dimension (that is we rearrange the spikes in time). Such permuted spike trains are then passed to the next layer, and we continue this process until the output layer. The permutation we apply after each layer is random, based on the current seed.

Generalization to TPP: Conceptually, a permutation would require to 1) collect the spike trains and 2) rearrange them in temporal dimension. These two correspond to two phases in our TPP neuron. In the second phase, TPP is producing the spike train, and the total number of spikes is given by Theorem 1. (c). Furthermore, at any given time step, there is a non-zero probability to have a spike (as long as there is non-zero residue voltage). Since in the end, produced spike trains will have "predetermined" number of spikes and there is a possibility to have a spike at every given time, it is as if TPP was acting as a permutation on a hypothetical spike train with the same number of spikes.

We hope this clarification makes the connection between permutation and TPP neurons more explicit, but please also kindly refer to Section 3.2 in our paper.

Weaknessess

Temporal misalignment in baselines We consider the situation of an ANN neuron with ReLU activation and how its output is approximated with a vanilla spiking neuron and our proposed TPP neuron.

Proposition Consider an ANN neuron with ReLu activation and let activation values follow distribution with PDF $p(x)$. Let further $\theta>0$ be the threshold of the corresponding vanilla SNN neuron. Then:

• The expectation of a spike output at step $t=1$ of a vanilla spiking neuron is $\int_{\theta}^\infty p(x)dx$.

• The expectation of having a spike at step $t=1$ of a TPP neuron is $\int_{0}^\infty p(x)dx$ which is the same as the expectation of the output of the ANN neuron with ReLU activation.

In particular, this result shows that TPP neurons with their designed probabilistic spiking approximate the output of the ANN neurons as starting with the first time step, while for vanilla spiking neurons this approximation is coarser (see also Figures 8 for an empirical evidence of the above Proposition). Our Theorem 1 (b) then shows how this approximation evolves throughout the rest of the simulation time.

Different spike counts Note that in Figure 4 and Figure 7 the distribution of membrane potential is presented for time steps 1 and 2. This is to empirically show that baselines do not have enough membrane potential to produce spikes in the first time steps, which eventually causes approximation errors as we discussed above. However, in Figures 5 nad 8, we are comparing spike counts for latencies 8 and higher. In particular, baselines are producing more spikes, however, these spikes trains are not "evenly" or "optimally" positioned in time, hence the gap in performance compared to our method.

We hope that this discussion contributes to the clarity of the paper, but we will incorporate your objections in the revised manuscript.

We thank the reviewer for their time in assessing our paper and constructive comments.

Essential References Not Discussed:

Thank you for pointing these out, we will include them in the revised manuscript.

Questions For Authors:

1. We reported in Tables 8-10 the number of spikes as, in general, this is a fair "measure" to use when comparing various methods for the same architectures and latency, on neuromorphic hardware. However, we provide further tables that compute the energy consumption on specialized hardware. In particular, our approach follows Merolla, et al. "A million spiking neuron integrated circuit with a scalable communication network and interface" where synaptic operations were used to calculate the energy, as this approach has been adapted in recent relevant literature. To estimate the energy consumptions per one SOP (FLOP), we use the values for the neuromorphic processor Qiao et al. "A reconfigurable on-line learning spiking neuromorphic processor comprising 256 neurons and 128K synapses". To be fair, we also calculated potential MAC operations in our and baseline models coming from the first layer, as the sample images use constant encoding.

In the following table (please refer to Table 17), we compare the energy consumption for some of the baseline methods and our approach, together with the approximated ANN energy consumption. We compare the energy consumption based on the accuracy. As our method reaches the ANN accuracy using much lower number of SOPs, it provides a more energy efficient alternative. However, we acknowledge that the stochastic component of our method can induce further energy consumption, but we were not able to test for that. We will provide full tables and more comparison results in the updated manuscript.

https://drive.google.com/drive/folders/1sbhqT9Nabl8BUIDs9dt8OFpsz1sMZK6w

Generalization We applied our proposed method on a simple Reinforcement Learning (DQN) example. We compared performance of the baseline QCFS (L=8) and TPP on CartPole task with over 20 evaluation epochs. The following table reports the average episode length (with standard deviation in parentheses) for different time horizons (T). A higher episode length indicates better performance, with a maximum possible score of 500 steps. The baseline struggles at T=4 but improves with higher latency, while TPP achieves significantly better performance at lower T values and reaches optimal performance faster.

Reinforcement Learning

In our current work in progress, we consider generalization of TPP neurons that are specific to approximate various non-linear activation functions, as a first step towards conversion of various ANN architectures. In particular, design that we are currently exploring is the following (please see the following Figure for a visual explanation).

activation function

We consider a general nice enough activation function $f$, and a sequence of thresholds $\theta_1,\dots,\theta_T$, chosen in such a way that the steps with length $\theta_i$ and height $s=1$ approximate the function $f$ in an ``optimal'' way (we do not discuss how to choose $\theta_i$ and what optimality would actually mean). If we denote by $\Theta_t:=\sum_{i=1}^t\theta_i$, the modified spiking mechanism of our TPP neuron becomes $s[t] = B(\frac{v[t-1]}{\Theta_{T-t+1}})$, $v[t+1] = v[t]-s[t]\theta_t$. We note that the mechanism generalizes our TPP in that for ReLU activation we had $\theta_t=\theta$ for all $t$. This situation requires more sophisticated approach for theoretical insights, and we decided to keep it separate from our submission.

3. We provide the required tests in the following tables (please refer to Tables 3-15). The results confirm that the reported changes in TPPs spiking changes are meaningful.

https://drive.google.com/drive/folders/1sbhqT9Nabl8BUIDs9dt8OFpsz1sMZK6w