One-Hot Multi-Level LIF Spiking Neural Networks for Enhanced Accuracy-Latency Tradeoff

Q4: In your energy calculations, do you factor in the weights associated with different spike lanes? For instance, if your M-LIF model operates with T=1 and S=3, making it equivalent to a conventional SNN with T=4, should the energy consumption not scale proportionally by the number of spikes (i.e., multiply by 4)?

A4: Thank you for raising this question. We appreciate the opportunity to clarify our energy calculations.

In one-hot M-LIF SNNs, the weights are indeed shared across spike lanes. One-hot M-LIF does not introduce overhead in that regard.

With respect to energy scaling proportionally, a ($T=1$, $S=3$) one-hot M-LIF SNN would be equivalent to a ($T=7$) traditional SNN in terms of overall spike activity in the worst-case scenario where all neurons fire during every timestep. However, it’s important to note that the energy consumption is not solely based on the number of timesteps but is also determined by the firing rate, which is influenced by the learned threshold and leakage parameters of the neuron model during training. Therefore, the energy consumption does not scale proportionally with the number of spikes alone. Instead, it depends on the firing rates observed during inference. Our energy calculations take into account these firing rates, ensuring a more accurate estimation of computational energy. This reflects the actual operational conditions and provides a better understanding of the energy efficiency improvements offered by the one-hot M-LIF model.

We hope this explanation addresses your concern and makes the relationship between timesteps, firing rates, and energy consumption in our model clearer. Thank you again for your insightful question.

References

[1] Xiao et al., "Fast and accurate classification with a multi-spike learning algorithm for spiking neurons." In IJCAI 2019.

[2] Miao et al., "A supervised multi-spike learning algorithm for spiking neural networks." In IJCNN 2018.

[3] Wang et al., "MT-SNN: Enhance Spiking Neural Network with Multiple Thresholds." In ArXiv 2023.

[4] Feng et al., "Multi-Level Firing with Spiking DS-ResNet: Enabling Better and Deeper Directly-Trained Spiking Neural Networks." In IJCAI 2022.

[5] Wang et al., “Bursting Spikes: Efficient and High-performance SNNs for Event-based Vision.” In arXiV 2023.

[6] Li et al., “Efficient and Accurate Conversion of Spiking Neural Network with Burst Spikes”. In IJCAI 2022.

[7] Datta et al., "Can we get the best of both Binary Neural Networks and Spiking Neural Networks for Efficient Computer Vision?" In ICLR 2024.

[8] Lee et al., "Reconfigurable Dataflow Optimization for Spatiotemporal Spiking Neural Computation on Systolic Array Accelerators." In ICCD 2020.

[9] Lee et al., "Parallel Time Batching: Systolic-Array Acceleration of Sparse Spiking Neural Computation." In HPCA 2022.

[10] Horowitz, “1.1 computing’s energy problem (and what we can do about it).” In ISSCC 2014.

Q3: Lack of Hardware Implementation Discussion: The paper does not address how the proposed M-LIF model can be supported by existing or future hardware architectures. It remains unclear how the model can be practically deployed in neuromorphic hardware. How does your proposed M-LIF model align with current or emerging hardware architectures? Could you provide insights into the feasibility and potential challenges of implementing M-LIF neurons in hardware to achieve the claimed energy efficiency improvements?

A3: Thank you for your insightful feedback. We appreciate the opportunity to elaborate on the hardware implementation aspects of the proposed M-LIF model.

The M-LIF model is designed to be adaptable to many existing hardware architectures. For example, previous work has shown the feasibility of leveraging systolic arrays for performing SNN inference using a small number of timesteps [8-9]. Adapting these methods to one-hot M-LIF SNNs would involve storing the exponent of the spike lane output $\in$ {$0, 2^i$}, where $-1 < i < S-1$ and $S$ is the number of spike lanes, instead of traditional single-bit activation.

Our experiments indicate significant benefits from using up to four spike lanes ($S=4$) with accuracy improvements saturating beyond that point (as detailed in the ablation study in response to Q2 for Reviewer egRh). Consequently, storage requirements would increase by only 1 or 2 additional bits per activation compared to traditional SNNs. This minimal memory overhead allows us to achieve higher accuracy while also minimizing the number of timesteps, which scales linearly with the number of FP32 weights and membrane potential high-energy memory loads.

During computations, instead of masking the weight, we perform an INT8 addition on the exponent, as shown in Appendix A.1. This addition is approximately $30\times$ cheaper than the multiplication required in ANNs. For hardware implementation of one-hot M-LIF thresholding, a simple priority encoder can be used to return the first non-zero bit seen from the most significant bit of the membrane potential after accumulation.

Moreover, it is worth noting that current neuromorphic hardware such as Loihi 2 supports multi-level SNNs, as mentioned by Reviewer NG6F. Therefore, one-hot M-LIF SNNs can indeed be mapped onto such neuromorphic hardware, ensuring their practical applicability and alignment with existing and emerging hardware architectures.

References

[1] Xiao et al., "Fast and accurate classification with a multi-spike learning algorithm for spiking neurons." In IJCAI 2019.

[2] Miao et al., "A supervised multi-spike learning algorithm for spiking neural networks." In IJCNN 2018.

[3] Wang et al., "MT-SNN: Enhance Spiking Neural Network with Multiple Thresholds." In ArXiv 2023.

[4] Feng et al., "Multi-Level Firing with Spiking DS-ResNet: Enabling Better and Deeper Directly-Trained Spiking Neural Networks." In IJCAI 2022.

[5] Wang et al., “Bursting Spikes: Efficient and High-performance SNNs for Event-based Vision.” In arXiV 2023.

[6] Li et al., “Efficient and Accurate Conversion of Spiking Neural Network with Burst Spikes”. In IJCAI 2022.

[7] Datta et al., "Can we get the best of both Binary Neural Networks and Spiking Neural Networks for Efficient Computer Vision?" In ICLR 2024.

[8] Lee et al., "Reconfigurable Dataflow Optimization for Spatiotemporal Spiking Neural Computation on Systolic Array Accelerators." In ICCD 2020.

[9] Lee et al., "Parallel Time Batching: Systolic-Array Acceleration of Sparse Spiking Neural Computation." In HPCA 2022.

[10] Horowitz, “1.1 computing’s energy problem (and what we can do about it).” In ISSCC 2014.

Q2: Performance: Although the paper presents extensive experiments, the results on datasets like CIFAR10 and CIFAR100 under equivalent timesteps (e.g., T=1, S=3 compared to traditional SNNs with T=4) show suboptimal performance. This raises concerns about the effectiveness of the proposed method in improving accuracy.

A2: Thank you for your valuable observation. We acknowledge that for the Transformer-2-512 architecture on CIFAR10 and CIFAR100, the performance of the one-hot M-LIF method ($T=1$, $S=3$) is comparable to the SNN method ($T=4$) ($95.4$% vs $95.6$% on CIFAR10, $78.2$% vs $78.4$% on CIFAR100). However, our primary goal with one-hot M-LIF was to reduce the number of timesteps $T$ during SNN inference while maintaining accuracy and preserving the low spike rates characteristic of traditional SNNs.

By reducing the timesteps, we aim to decrease the energy overhead associated with multi-timestep processing. The need for multi-timestep processing poses a significant challenge for widespread SNN deployment, as it often results in increased memory storage and access requirements, which can exceed the computational costs. Therefore, the ($T=1$, $S=3$) one-hot M-LIF SNN offers a more optimal tradeoff compared to the ($T=4$) SNN because it achieves comparable accuracy while avoiding the memory access overhead that scales linearly with timesteps.

For further illustration, please refer to our response to Q4 for Reviewer bytk, where we provide an example of the impact of multi-timestep processing on the total energy consumption of VGG16 for ImageNet, based on the memory energy model proposed in [7]. Additionally, for the larger and more challenging ImageNet dataset, evaluations of spike-driven Transformer-8-512 show that ($T=1$, $S=3$) one-hot M-LIF SNN outperforms ($T=4$, $S=1$) traditional SNN in terms of accuracy.

Lastly, our approach demonstrates improved performance for dynamic vision classification tasks on DVS-CIFAR10, as shown in Table 3 of our submission. It surpasses traditional SNNs in accuracy and preserves accuracy better than traditional SNNs in low-latency, memory-efficient regimes.

References

[1] Xiao et al., "Fast and accurate classification with a multi-spike learning algorithm for spiking neurons." In IJCAI 2019.

[2] Miao et al., "A supervised multi-spike learning algorithm for spiking neural networks." In IJCNN 2018.

[3] Wang et al., "MT-SNN: Enhance Spiking Neural Network with Multiple Thresholds." In ArXiv 2023.

[4] Feng et al., "Multi-Level Firing with Spiking DS-ResNet: Enabling Better and Deeper Directly-Trained Spiking Neural Networks." In IJCAI 2022.

[5] Wang et al., “Bursting Spikes: Efficient and High-performance SNNs for Event-based Vision.” In arXiV 2023.

[6] Li et al., “Efficient and Accurate Conversion of Spiking Neural Network with Burst Spikes”. In IJCAI 2022.

[7] Datta et al., "Can we get the best of both Binary Neural Networks and Spiking Neural Networks for Efficient Computer Vision?" In ICLR 2024.

[8] Lee et al., "Reconfigurable Dataflow Optimization for Spatiotemporal Spiking Neural Computation on Systolic Array Accelerators." In ICCD 2020.

[9] Lee et al., "Parallel Time Batching: Systolic-Array Acceleration of Sparse Spiking Neural Computation." In HPCA 2022.

[10] Horowitz, “1.1 computing’s energy problem (and what we can do about it).” In ISSCC 2014.

Q1: Novelty: The proposed M-LIF neuron model closely resembles existing concepts such as multi-spike, burst spike, and multi-threshold neurons. While the approach of restricting outputs to powers of two introduces some variation, it does not fundamentally differentiate the model from prior research, potentially limiting its originality. And the author did not discuss any of those methods. Could you elaborate on the fundamental differences between your proposed M-LIF neuron model and existing models such as multi-spike, burst spike, and multi-threshold neurons?

A1: Thank you for your thoughtful question. We appreciate the opportunity to further clarify the novelty of our proposed one-hot M-LIF neuron model and its fundamental differences from existing approaches like multi-spike, burst-spike, and multi-threshold neurons. We have now added a paragraph in Section 2.5 of our submission to better contextualize these methods relative to our work.

Multi-spike [1,2] and multi-threshold [3,4] neurons increase activation precision by introducing uniform activation quantization into SNNs. This leads to higher computational costs, particularly in terms of energy consumption, as it breaks the multiplication-free nature of traditional SNNs and often requires multiplications or multiple successive additions to handle the increased precision. Burst-spike models [5,6] work by increasing the spike rate within a single timestep, which also enhances precision. However, burst-spike methods often do not scale down to unit-timestep processing, as demonstrated in [5], where results are provided for at least $T=8$ timesteps on ImageNet. Memory energy scales linearly with the number of timesteps, with single memory access costs often being $10\times$ higher than single compute energy costs [10]. In contrast, our one-hot M-LIF neuron reduces the timestep requirement to $T=1$, which significantly cuts down on the associated memory access overhead.

Our one-hot M-LIF neuron utilizes a single threshold (and its powers of two) to ensure that only one spike lane fires per timestep. This restriction is learned during training, making it a unique feature not present in the multi-spike and multi-threshold approaches. This power-of-two encoding enables scaling weights efficiently with a single INT8 addition to their exponent values during membrane potential updates, avoiding the need for multiplications or complex additions that are typical in [1,2,3,4] models. Please also refer to the answer to Q1 for Reviewer NG6F for a comparison between one-hot M-LIF SNNs and multi-threshold [3,4] SNNs.

The unique features of our approach are also demonstrated in our experimental results. Unlike burst-spike [5,6] models that require multi-timestep processing, our model achieves high accuracy with unit-timestep processing, which is particularly beneficial for both energy efficiency and scalability. We provide comparisons against unit-timestep SNNs in Table 1 of our manuscript for static image classification. Finally, our method is scalable to more complex tasks like ImageNet classification and spike-driven transformer architectures.

References

[1] Xiao et al., "Fast and accurate classification with a multi-spike learning algorithm for spiking neurons." In IJCAI 2019.

[2] Miao et al., "A supervised multi-spike learning algorithm for spiking neural networks." In IJCNN 2018.

[3] Wang et al., "MT-SNN: Enhance Spiking Neural Network with Multiple Thresholds." In ArXiv 2023.

[4] Feng et al., "Multi-Level Firing with Spiking DS-ResNet: Enabling Better and Deeper Directly-Trained Spiking Neural Networks." In IJCAI 2022.

[5] Wang et al., “Bursting Spikes: Efficient and High-performance SNNs for Event-based Vision.” In arXiV 2023.

[6] Li et al., “Efficient and Accurate Conversion of Spiking Neural Network with Burst Spikes”. In IJCAI 2022.

[7] Datta et al., "Can we get the best of both Binary Neural Networks and Spiking Neural Networks for Efficient Computer Vision?" In ICLR 2024.

[8] Lee et al., "Reconfigurable Dataflow Optimization for Spatiotemporal Spiking Neural Computation on Systolic Array Accelerators." In ICCD 2020.

[9] Lee et al., "Parallel Time Batching: Systolic-Array Acceleration of Sparse Spiking Neural Computation." In HPCA 2022.

[10] Horowitz, “1.1 computing’s energy problem (and what we can do about it).” In ISSCC 2014.

I appreciate the authors' detailed response and the effort put into addressing the concerns raised. However, after careful consideration and review of other reviewers' opinions, I maintain my original score.

Q3: What unique features distinguish the proposed model from existing models in the literature, as well as from available neuromorphic hardware and open-source SNN projects? How does the proposed model perform in comparison to these alternatives?

A3: Thank you for your thoughtful question. We appreciate the opportunity to further clarify the unique features of our proposed model.

Our one-hot M-LIF neuron utilizes one-hot (power-of-two) encoding of activations, meaning that only one of the $S$ spike lanes fires per timestep. This one-hot constraint is applied and learned during training, which sets our model apart from previous binary activation SNNs [4], [5], [6], [8], which do not incorporate such a learned one-hot activation scheme. This encoding approach has several advantages, as it ensures activation precision without the need for multiplication operations. Instead, the neuron’s output corresponds to powers of two, which can be efficiently used to modify the exponents of FP32 weights with a single INT8 addition during membrane potential updates, offering computational efficiency without sacrificing accuracy.

Additionally, our work includes a comprehensive evaluation across different architectures, including both convolutional neural networks and more complex, spike-driven transformer models [5]. This is in contrast to previous works that focused on specific networks or architectures.

Our model is distinct from the work of authors like those in [7], who introduced uniform activation quantization to offload multi-timestep processing and improve activation precision per timestep. However, this quantization approach introduces additional computational overhead and breaks the multiplication-free property of SNNs. Table 1 of our manuscript compares our one-hot M-LIF SNN to binary-activated SNNs that aim to minimize the number of timesteps while preserving accuracy. We also provide a comparison to log-quantized ANNs in Table 2, showing that our model, using unit-timestep processing, achieves competitive accuracy and energy efficiency. To clarify the performance differences with other methods, we have included the following comparison to [7] below:

As seen in this table, our one-hot M-LIF model (with 3 spike lanes) outperforms existing methods in terms of both accuracy and energy consumption, achieving an accuracy of $78.2$% while using only $0.478$ mJ of computational energy, a marked improvement over the Parallel-MT and Cascade-MT models, which achieve $73.89$% and $74.80$% accuracy, respectively, but consume much higher computational energy.

IJCAI22 [10] also proposes introducing uniform activation quantization into SNNs but only evaluates on small datasets (CIFAR10, DVS-CIFAR10). We see that our one-hot M-LIF model significantly outperforms their results for both CIFAR10 and DVS-CIFAR10 datasets.

Our one-hot M-LIF SNN achieves higher accuracy ($95.4$% vs. $94.25$%) on CIFAR10 and $14$% higher accuracy on DVS-CIFAR10 ($84.7$% vs. $70.36$%) when compared to [10], showcasing superior performance across both static and dynamic datasets.

Finally, we also note that AAAI22 [1] focuses exclusively on speech recognition tasks, whereas our experimental results are centered on static and dynamic image classification, providing broader applicability and relevance across a variety of tasks.

Please see references in first comment

Q2: Regarding the energy efficiency claims in the weaknesses section, could the authors elaborate on the assumptions made for the energy comparisons? How might these comparisons change if memory access energy is included?

A2: Thank you for your insightful comments. We appreciate the importance of contextualizing energy comparisons and recognize that memory access can significantly impact energy consumption, particularly in SNNs. We will clarify the assumptions behind our energy comparisons. Below is a detailed response to your concerns.

Energy Comparisons and Assumptions

As noted in Section 4, energy modeling for SNNs has been performed using the methodology adopted from prior works in the field [4-8]. In the absence of a specified hardware architecture, we focus on computational energy as a metric that is often used in SNN papers. Specifically, we calculate computational energy based on the number of multiply-accumulate (MAC) or add-accumulate (AC) operations, the spike rates per layer, and the unit MAC/AC energy for a 45nm technology node, where $E_{mac} = 4.6$ pJ and $E_{ac} = 0.9$ pJ [9]. Therefore, the computational energies in Tables 1, 2, and 3 reflect both the number of MAC/AC operations and the average firing rates per layer, providing a reasonable estimate of energy consumed by non-zero operations.

Memory Access Energy

We acknowledge in our manuscript that memory access energy can be orders of magnitude higher than compute energy, particularly when dealing with multi-timestep SNNs. As you rightly pointed out, memory access energy plays a crucial role in energy consumption, especially since SNNs require more memory accesses than ANNs due to multi-timestep processing. This is indeed important when comparing SNNs and ANNs and motivates the goal of our unit-timestep one-hot M-LIF approach. By reducing the number of timesteps during SNN inference while improving accuracy and maintaining the low-spike rates of traditional SNNs, we can decrease the multi-timestep processing energy overhead. In our manuscript, we compare unit-timestep one-hot M-LIF SNNs with ANNs in Tables 1 and 2. There, the memory access patterns for ANNs are identical to those in unit-timestep one-hot M-LIF SNNs with the latter benefitting from a smaller memory footprint due to the lower bit-width activations.

Impact of Memory Access Energy

Although defining a precise memory architecture requires detailed assumptions regarding weight, input, and partial output reuse, we have included an analysis based on the memory energy model provided in Appendix A.8.2 of [8] for convolutional neural networks. This model allows us to estimate the impact of memory energy by incorporating per-layer spike rates, which we obtained from multiple prior works [4], [6], [8].

For our analysis, we focused on VGG16 trained on ImageNet, as prior works [4], [6], [8] provide detailed per-layer spike rates, and we used the same 45nm technology for energy modeling, allowing for consistent energy comparisons with previous works [4] and [6]. The table below illustrates the impact of memory access energy on the overall energy comparison.

As shown above, one-hot M-LIF SNNs consume $33\times$ less total energy than ANNs, while maintaining higher accuracy (up to $> 3$%) than prior unit-timestep SNN works. We highlight the advantage of using unit-timestep processing in this table since we achieve higher accuracy compared to [7] while consuming $2.5\times$ less total energy, the majority of which stems from the memory energy overhead of multi-timestep (T=5) processing.

We hope this provides a clearer and more contextualized view of our energy comparisons. Finally, we have added this analysis for memory access energy impact in Appendix A.5 of our manuscript where we also include a graph illustrating a comparison of per layer spike rates for VGG16 ImageNet against [4], [6], [8].

Q1: The manuscript lacks comparisons with other methods that use graded (weighted) spikes. The idea of utilizing multibit outputs (one-hot encoded or otherwise) is not novel, as it has been previously explored, for instance in [1], and is a feature available in neuromorphic hardware like Loihi 2 [2], as well as in commonly used SNN libraries like snnTorch [3]. I recommend that the authors provide a discussion highlighting the unique features of the proposed model in comparison to these existing methods and tools.

A1: Thank you for your thoughtful feedback and for bringing relevant works to our attention. We appreciate the opportunity to clarify the distinction between our approach and existing methods that use graded (weighted) spikes, as well as to highlight the unique features of our proposed model. We have added a discussion in Section 2.5 that compares our work with the methods mentioned.

In AAAI22 (Figure 4) [1], the authors essentially propose uniform activation quantization as a way to alleviate the burden of multi-timestep processing while preserving better activation precision per timestep. However, this approach essentially breaks the multiplication-free property of traditional SNNs by introducing uniform quantization, which leads to an increase in spike rates. This is a key difference from our approach as explained in section 4 of our manuscript. Our one-hot M-LIF neuron utilizes a one-hot (power-of-two) encoding of activations, ensuring that only one of the S spike lanes is activated per timestep. This constraint is applied and learned during training, unlike in [1], where no similar restriction is enforced. The outputs from our neurons correspond to powers of two, enabling efficient computation. Specifically, these powers-of-two outputs allow us to modify the exponents of FP32 weights through a single INT8 addition during membrane potential updates, making our approach energy-efficient and maintaining the multiplication-free property of SNNs. In contrast, methods such as those in [1] that use uniformly quantized activations (e.g., 3 or 4 bits) still require multiplications, which can be less efficient in terms of both computation and energy consumption.

While works [2] and [3] support multi-bit SNN operations with broader functionality compared to our proposed one-hot encoding, we would like to emphasize the unique computational efficiency offered by one-hot M-LIF neurons. The output range of one-hot M-LIF neurons represents a subset of uniformly quantized activations, so our approach can provide distinct efficiency benefits, particularly when hardware is specifically designed to leverage these properties, as discussed in our paper. Nonetheless, the one-hot M-LIF SNNs can be effectively mapped onto existing neuromorphic hardware architectures, thereby demonstrating both practical applicability and strong alignment with current technological advancements.

We hope this clarifies the unique aspects of our model in comparison to existing approaches and tools.

References

[1] Ponghiran et al., “Spiking Neural Networks with Improved Inherent Recurrence Dynamics for Sequential Learning.”. In AAAI 2022.

[2] M. Davies et al., "Advancing Neuromorphic Computing With Loihi: A Survey of Results and Outlook," in Proceedings of the IEEE 2021.

[3] https://snntorch.readthedocs.io/en/latest/index.html

[4] Chowdhury et al., “Towards ultra low latency spiking neural networks for vision and sequential tasks using temporal pruning.” In ECCV 2022.

[5] Yao, et al., “Spike-driven transformer.” In NeurIPS 2023.

[6] N. Rathi et al., "DIET-SNN: A Low-Latency Spiking Neural Network With Direct Input Encoding and Leakage and Threshold Optimization." In TNNLS 2023.

[7] Wang et al., "MT-SNN: Enhance Spiking Neural Network with Multiple Thresholds." In ArXiv 2023.

[8] Datta et al., "Can we get the best of both Binary Neural Networks and Spiking Neural Networks for Efficient Computer Vision?" In ICLR 2024.

[9] Horowitz, “1.1 computing’s energy problem (and what we can do about it).” In ISSCC 2014.

[10] Feng et al., "Multi-Level Firing with Spiking DS-ResNet: Enabling Better and Deeper Directly-Trained Spiking Neural Networks." In IJCAI 2022.

Q3: For the DVS-CIFAR10 dataset, as the authors add an additional multi-level input layer encoding, could the authors provide a latency comparison between the M-LIF model and other existing models?

A3: Thank you for your insightful question regarding the latency comparison between our one-hot M-LIF SNN model and existing models, specifically in relation to the DVS-CIFAR10 dataset.

Both traditional SNNs and our one-hot M-LIF SNNs require a pre-processing step done once per inference to convert the asynchronous event stream into frames prior to running model inference. This step is necessary for both models to enable effective processing of the input data. In our approach, we use a multi-level input layer encoding that allows us to reduce the number of timesteps required for inference. Specifically, we are able to reduce the number of timesteps from $T=10$ to $T=5$, which effectively halves the overall latency. With this reduction, our model achieves an accuracy of $83.3$% using $S=4$ spike lanes, while traditional SNNs using $T=10$ timesteps achieve a slightly lower accuracy of $83.1$%, as reported in Table 3 of our submission. It is important to note that the pre-processing overhead introduced by the multi-level input layer encoding is minimal compared to the latency incurred by performing multi-timestep inference. As a result, the overall latency of our model remains competitive, with the benefit of achieving higher accuracy at a lower number of timesteps.

In summary, while the additional multi-level input layer encoding introduces a minor overhead, its impact on latency is negligible when compared to the significant latency savings gained from reducing the number of timesteps. Therefore, our one-hot M-LIF SNN model offers a more efficient solution, achieving comparable or slightly better performance with reduced computational delay.

References

[1] Jaehyeon Moon et al., “Instance-Aware Group Quantization for Vision Transformers.” In CVPR 2024

[2] Yefei He et al., “BiViT: Extremely Compressed Binary Vision Transformers.” In ICCV 2023

[3] Yanjing Li et al., “Q-ViT: Accurate and Fully Quantized Low-bit Vision Transformer.” In NeurIPS, 2022

[4] Yao, et al., “Spike-driven transformer.” In NeurIPS 2023.

[5] Chowdhury et al., “Towards ultra low latency spiking neural networks for vision and sequential tasks using temporal pruning.” In ECCV 2022.

[6] Horowitz, “1.1 computing’s energy problem (and what we can do about it).” In ISSCC 2014.

Q2: Although the authors provide some results for models with varying numbers of spike lanes, an additional ablation study would be beneficial. Specifically, the authors could conduct experiments with a fixed number of time steps, varying the number of spike lanes (S) from 1 to an upper bound (e.g., 8), and plot the accuracy versus S. This would help identify the point of saturation. Additionally, a complementary study could be performed by fixing the number of spike lanes and varying the number of time steps from 1 to an upper bound (e.g., 10). Such an analysis would offer a clearer comparison of the impact of spike lanes versus the time steps on model performance

A2: Thank you for your valuable suggestion regarding the ablation study. We appreciate your interest in further understanding the effects of varying spike lanes and time steps on model performance.

In response to your suggestion, we have conducted an ablation study where we varied the number of spike lanes ($S$) while keeping the number of timesteps ($T$) fixed at $3$ for VGGSNN on DVS-CIFAR10, and we provide the results below. Our findings indicate that as the number of spike lanes increases from $S=1$ to $S=4$, there is a significant improvement in accuracy, after which performance saturates:

Additionally, when we increase the number of timesteps to $T=5$, we see a similar performance saturation around $4$ timesteps. We also note that the accuracy improvement due to spike lanes becomes less pronounced, but the overall accuracy ceiling improves as shown below:

From these results, we conclude that both timesteps and spike lanes contribute to better model performance. We also note that multi-timestep processing introduces significant energy overhead, particularly with respect to memory access. As noted in previous works [5], memory access energy for each timestep can be up to $10\times$ higher than the energy required for FP32 additions [6]. This increase in memory energy makes traditional SNNs less efficient than ANNs when the performance gains from additional timesteps do not outweigh the energy costs. Nevertheless, we include additional results below to show one-hot M-LIF SNN accuracy when $T > 1$, showing improved accuracy (but at the energy cost of multi-timestep).

We hope this additional analysis addresses your request and further clarifies the trade-offs between spike lanes and timesteps. We have also added this ablation study to Appendix A.4 of our manuscript.

References

[1] Jaehyeon Moon et al., “Instance-Aware Group Quantization for Vision Transformers.” In CVPR 2024

[2] Yefei He et al., “BiViT: Extremely Compressed Binary Vision Transformers.” In ICCV 2023

[3] Yanjing Li et al., “Q-ViT: Accurate and Fully Quantized Low-bit Vision Transformer.” In NeurIPS, 2022

[4] Yao, et al., “Spike-driven transformer.” In NeurIPS 2023.

[5] Chowdhury et al., “Towards ultra low latency spiking neural networks for vision and sequential tasks using temporal pruning.” In ECCV 2022.

[6] Horowitz, “1.1 computing’s energy problem (and what we can do about it).” In ISSCC 2014.

Q1: Given that the proposed spike lanes share similarities with quantized (S-bit activation, S is the number of spike lanes) models, the authors should include additional comparisons, both in terms of accuracy and FLOPs/energy consumption, with state-of-the-art quantization approaches such as [1][2][3]. This would provide a more comprehensive and fair evaluation of the method's performance

A1: Thank you for your thoughtful suggestion and for highlighting works [1-3]. We agree that a comprehensive comparison is essential for a robust evaluation of the method. However, it is important to clarify the distinctions between our work and [1-3].

Papers [1] and [3] employ uniform quantization techniques, whereas our approach utilizes one-hot encoding, a critical difference that provides additional hardware benefits and preserves the multiplication-free property of spiking neural networks (SNNs). This distinction is significant because while uniform quantization introduces multiplications—which increase computational complexity and require additional hardware resources—our method avoids this, offering a more efficient solution for SNNs.

Paper [2] focuses on quantizing weights but does not address activation quantization, which is a core aspect of our work. Our approach centers on the quantization of activations through one-hot (power-of-two) encoding, which necessitates modifications to both the neuron model (one-hot M-LIF) and the surrogate gradients used during training. This allows us to achieve energy-efficient, high-accuracy performance in low-latency, memory-efficient regimes (i.e., unit-timestep processing), without sacrificing the fundamental property of SNNs that distinguishes them from traditional artificial neural networks (ANNs)—the ability to perform computations without multiplications.

Furthermore, we believe that comparing our method directly with the works you mentioned—[1], [2], and [3]—is not entirely aligned with the scope of our paper. These works focus on optimizing quantization for vision transformer architectures, specifically minimizing the impact of quantization on the attention mechanism. In contrast, our work is focused on optimizing SNNs, particularly through the novel application of one-hot encoding to activations, which enhances both the accuracy and energy efficiency of SNNs.

In our evaluation, we compare our approach against state-of-the-art spike-driven transformer architectures [4], demonstrating significant improvements in performance, especially for large and challenging datasets such as ImageNet. Additionally, we have conducted a direct comparison between unit-timestep one-hot M-LIF SNNs and log-quantized artificial neural networks (LQ-ANNs). Our results show that one-hot M-LIF SNNs either match or outperform LQ-ANNs in both accuracy and inference energy efficiency. Specifically, for CIFAR-10, the performance and energy efficiency are comparable, while for CIFAR-100, our method is up to 54% more energy-efficient than LQ-ANNs, with similar accuracy. These findings, particularly the superior scalability of one-hot M-LIF SNNs on large datasets such as ImageNet, underline the strengths of our approach and the distinct contributions we make to the field of SNNs.

We hope this explanation clarifies the distinctions between our work and the approaches you suggested and highlights the specific contributions of our research to the field of spiking neural networks.

References

[1] Jaehyeon Moon et al., “Instance-Aware Group Quantization for Vision Transformers.” In CVPR 2024

[2] Yefei He et al., “BiViT: Extremely Compressed Binary Vision Transformers.” In ICCV 2023

[3] Yanjing Li et al., “Q-ViT: Accurate and Fully Quantized Low-bit Vision Transformer.” In NeurIPS, 2022

[4] Yao, et al., “Spike-driven transformer.” In NeurIPS 2023.

[5] Chowdhury et al., “Towards ultra low latency spiking neural networks for vision and sequential tasks using temporal pruning.” In ECCV 2022.

[6] Horowitz, “1.1 computing’s energy problem (and what we can do about it).” In ISSCC 2014.

Q2: How does M-LIF differ from quantization networks? Specifically, does an M-LIF with S=K operate equivalently to a K-bit quantized neural network (QNN) with quantized activation values?

A2: Thank you for this insightful question. While $K$-bit quantized neural networks represent activations using $2^K$ discrete values, they do not naturally extend to neuromorphic tasks, such as dynamic vision image classification. In contrast, a one-hot M-LIF spiking neural network (SNN) with $S=K$ employs $K$ power-of-two values to represent activations, while also extending to neuromorphic tasks, as demonstrated by our experiments on DVS-CIFAR10.

In our paper, we elaborate on these distinctions and similarities in Sections 2.4 and 3.2, where we discuss the relationship between log-quantized artificial neural networks (LQ-ANNs) and unit-timestep M-LIF SNNs. Additionally, Section 5.1.3 presents our evaluation comparisons against log-quantized ANNs. The key differences between LQ-ANNs and one-hot M-LIF SNNs are as follows: (1) during inference, neuron dynamics in M-LIF differ when $T>1$ due to their extensibility to neuromorphic tasks, and (2) during training, we learn threshold and leakage parameters for static tasks and employ back-propagation through time when $T>1$.

Q3: Table 1 shows that MLIF significantly increases computational energy consumption, which poses a drawback for MLIP. For instance, on CIFAR10, although MLIP-Transformer with S=3 and T=4 achieves the highest accuracy (only 0.3% higher than Spike-driven Transformer with S=1 and T=4), its computational energy usage is considerably higher. This trend is similarly observed on CIFAR100 and ImageNet

A3: Thank you for bringing attention to the computational energy consumption in one-hot M-LIF SNNs compared to traditional SNNs. Your observation is valid concerning the increased computational energy of the spike-driven transformer evaluated on CIFAR-10 when comparing scenarios with $S=3$, $T=4$ and $S=1$, $T=4$. Here, we acknowledge that while the one-hot constraint of the M-LIF SNN guarantees an upper bound on the number of spikes (ensuring that it will not exceed the worst-case scenario of a traditional SNN), there is no explicit guarantee of lower computational energy consumption compared to a traditional SNN with the same architecture and number of timesteps.

However, it is important to contextualize this increase in computational energy within the broader trade-off between computational and memory access energy. As noted, the memory access energy is typically much higher than the computational energy. For example, accessing a 8kB cache in 45nm technology consumes approximately 10pJ per 32-bit access, compared to just 0.9 pJ for an addition operation [4]. Multi-timestep processing in traditional SNNs incurs a significant increase in memory energy overhead, scaling linearly with the number of timesteps. In contrast, our one-hot M-LIF approach mitigates this overhead by reducing the reliance on multi-timestep processing, making it a more favorable solution from a holistic energy perspective.

Additionally, the observed increase in computational energy is not universally present across all datasets and architectures. For instance, in ResNet-20 on CIFAR10, our approach demonstrates lower computational energy consumption compared to higher-timestep SNNs (excluding memory access energy). Similarly, for the spike-driven transformer on ImageNet at $T=4$, $S=1$, we achieve a higher accuracy ($75.33$% vs $74.57$%) while consuming less computational energy than its higher-timestep counterpart. These examples underscore the broader benefits of our approach, particularly in scenarios where memory energy plays a dominant role in overall energy consumption. We are happy to discuss any further clarifications regarding these findings and trade-offs.

References

[1] Wang et al., "MT-SNN: Enhance Spiking Neural Network with Multiple Thresholds." In ArXiv 2023.

[2] Feng et al., "Multi-Level Firing with Spiking DS-ResNet: Enabling Better and Deeper Directly-Trained Spiking Neural Networks." In IJCAI 2022.

[3] Yao, et al., “Spike-driven transformer.” In NeurIPS 2023.

[4] Horowitz, “1.1 computing’s energy problem (and what we can do about it).” In ISSCC 2014.

Q1: The novelty of employing multi-level thresholds in SNNs appears limited, as similar concepts have been introduced in related works [1,2]. What distinguishes M-LIF from these existing approaches?

A1: Thank you for highlighting the prior works on multi-level thresholds in SNNs. We have now added a paragraph in Section 2.5 of our submission to better contextualize these methods relative to our work. Our proposed one-hot M-LIF neuron model addresses critical computational challenges inherent in approaches [1,2]. We will outline these distinctions below.

In arXiv23 [1], the authors introduce parallel and cascade multi-threshold (MT) neuron models to enhance activation precision per timestep and mitigate the burden of multi-timestep processing. The parallel-MT model uses multiple threshold values, summing spikes from all firing lanes at each timestep, while the cascade-MT model processes them sequentially. These approaches utilize uniform activation quantization, which breaks the multiplication-free characteristic of SNNs and significantly raises spike rates. In contrast, our one-hot M-LIF neuron maintains a single threshold value, leveraging its power-of-two multiples to fire only one spike lane per timestep. This one-hot constraint is incorporated and learned during training—a crucial distinction not found in [1]. The resulting neuron outputs, represented as powers-of-two, allow for efficient FP32 weight exponent updates via a single INT8 addition, whereas [1]’s uniformly quantized outputs (e.g., 4-bit) require multiplications. Furthermore, our evaluations span both convolutional neural networks and more complex, high-performing spike-driven transformer architectures. Unlike [1], we also compare with log-quantized ANNs, which exhibit similarities to our one-hot M-LIF SNNs when operating with a unit timestep. The authors of [1] limit their evaluations to VGG-9 on CIFAR100, using numbers from a 45nm technology for energy estimates. A direct comparison demonstrates that our approach achieves superior accuracy gains with less computational energy overhead:

IJCAI22 [2] similarly introduces uniform activation quantization, compromising the multiplication-free property of SNNs. Unlike our work, it does not impose constraints on input/output encoding during training, which is critical for one-hot behavior that enables preserving activation precision per timestep while maintaining computational simplicity. [2] employs individual thresholds that are not multiples of each other, resulting in more parameters. From an evaluation perspective, [2] focuses on residual networks without reporting results on larger datasets like ImageNet or on SOTA architectures, such as spike-driven transformers. [2] also does not offer comprehensive energy analyses to estimate the overhead of their approach. Below, we provide a comparison illustrating our method's superior performance on CIFAR10 and DVS-CIFAR10 datasets:

I agree with the other reviewers that the novelty of this paper is limited. I greatly appreciate the authors' efforts in their rebuttal. However, I have decided to maintain my original score.

Q3: I see some empirical comparisons of this method with log-quantized ANNs, and different modes of operations during training. However, during inference, the operations are almost similar. Even during training, I am not sure if the differences significantly affect the accuracy metrics

A3: Thank you for your thoughtful feedback regarding the comparison between log-quantized ANNs (LQ-ANNs) and our unit-timestep one-hot M-LIF SNNs. While it is true that layer computation during inference may appear structurally similar, one critical distinction lies in the degree of sparsity induced by the thresholds and weights learned during training. These differences in sparsity have a direct impact on the computational energy, aligning with our empirical observations. The variations in thresholds and weights between LQ-ANNs and M-LIF SNNs arise from their distinct training processes. Specifically, one-hot M-LIF SNNs are trained in a single-phase approach, whereas LQ-ANN training is typically conducted in two phases per epoch. The first phase involves recording a percentile value $\alpha_l$ for each layer’s input activation distribution using the full training dataset and full-precision inference. The second phase employs a straight-through estimator to approximate the gradient with respect to quantized activations. While there are certain similarities between one-hot M-LIF SNNs with unit timestep ($T=1$) and LQ-ANNs, key differences remain. As detailed in Equations (9-10) in Section 3.2 of our submission, these differences include variations in firing thresholds and final output value ranges, scaled by ​$\alpha_l$. Moreover, one-hot M-LIF SNNs offer the capability for the extension to sequential processing (i.e., $T>1$). Another significant distinction is the use of surrogate gradients in one-hot M-LIF SNNs during training. We have adapted these surrogate gradients from traditional SNN research to fit the specific characteristics of the one-hot M-LIF neuron model. This adaptation further differentiates our approach, enhancing the effectiveness and accuracy of our model relative to LQ-ANNs.

References:

[1] Lam et al. "Precision Batching: Bitserial Decomposition for Efficient Neural Network Inference on GPUs." In PACT 2021.

[2] Yuhang et al., “Additive powers-of-two quantization: An efficient non-uniform discretization for neural networks." In ICLR 2020.

[3] Shen et al., "Are Conventional SNNs Really Efficient? A Perspective from Network Quantization." In CVPR 2024.

[4] Datta et al., "Can we get the best of both Binary Neural Networks and Spiking Neural Networks for Efficient Computer Vision?" In ICLR 2024.

[5] Chowdhury et al., “Towards ultra low latency spiking neural networks for vision and sequential tasks using temporal pruning.” In ECCV 2022.

[6] Yao et al., “Spike-driven transformer.” In NeurIPS 2023.

[7] N. Rathi et al., "DIET-SNN: A Low-Latency Spiking Neural Network With Direct Input Encoding and Leakage and Threshold Optimization." In TNNLS 2023.

[8] Horowitz, “1.1 computing’s energy problem (and what we can do about it).” In ISSCC 2014.

Q1: While the paper has moderately strong empirical results, the proposed method is not much different from bit-serial activation quantized neural networks [1], where the spike lanes can be regarded as bits sequentially arriving and accumulating at each neuron. The overhead of the time steps is thus shifted to the spike lanes

A1: Thank you for highlighting this relevant aspect of quantized activation artificial neural networks (ANNs). We appreciate your reference to PACT21 [1], which indeed provides valuable insights into accelerating quantized neural networks using a bit-serial approach for weights and activations on GPUs. However, it is important to note that PACT21 focuses on accelerating quantized ANNs without delving into techniques for improving quantization-aware training. More specifically, it does not explore this learning aspect with spiking neural networks (SNNs).

Our work centers on enhancing the accuracy and efficiency of SNNs, which are inherently distinct from ANNs due to their use of the leaky integrate-and-fire (LIF) neuron model and their applicability to neuromorphic tasks. Our primary contribution lies in the one-hot (power-of-two) encoding of activations during both training and inference of SNNs, necessitating modifications to the neuron model (M-LIF) and the surrogate gradients employed during training. The M-LIF model achieves higher accuracy than conventional SNNs in low-latency, memory-efficient regimes (unit timestep) while preserving the multiplication-free nature of SNNs. This stands in contrast to alternative multi-threshold approaches that utilize uniform activation quantization, leading to increased spike rates and the reintroduction of multiplications. Additionally, unlike the bit-serial processing approach highlighted in [1], the spike lanes in M-LIF SNNs can be processed in parallel with a single addition to the weight exponent, as detailed in Section 4 of our submission. This parallelization is possible due to the predetermined one-hot nature of the activations, ensuring that only one "bit-position" (spike lane) is non-zero at any given timestep.

References:

[1] Lam et al. "Precision Batching: Bitserial Decomposition for Efficient Neural Network Inference on GPUs." In PACT 2021.

[2] Yuhang et al., “Additive powers-of-two quantization: An efficient non-uniform discretization for neural networks." In ICLR 2020.

[3] Shen et al., "Are Conventional SNNs Really Efficient? A Perspective from Network Quantization." In CVPR 2024.

[4] Datta et al., "Can we get the best of both Binary Neural Networks and Spiking Neural Networks for Efficient Computer Vision?" In ICLR 2024.

[5] Chowdhury et al., “Towards ultra low latency spiking neural networks for vision and sequential tasks using temporal pruning.” In ECCV 2022.

[6] Yao et al., “Spike-driven transformer.” In NeurIPS 2023.

[7] N. Rathi et al., "DIET-SNN: A Low-Latency Spiking Neural Network With Direct Input Encoding and Leakage and Threshold Optimization." In TNNLS 2023.

[8] Horowitz, “1.1 computing’s energy problem (and what we can do about it).” In ISSCC 2014.

Q5: I see only one comparison with one-time-step SNNs in Table 1. It would be better to add more comparisons, including the results from [4].

A5: Thank you for highlighting the need for additional comparisons. In response, we have updated our submission to include comparisons with the results from [4] for iso-architecture SNNs in Table 1. It is worth noting that Table 2 in ICLR24 [4] provides only one comparison with a unit-timestep SNN, specifically referring to an older (2021) arXiv version of what later became ECCV22 [5]. In contrast, our paper compares against the most recent and up-to-date reference, which outperforms the results reported in ICLR24 [4] for the VGG16 model on ImageNet. Finally, our paper includes an evaluation of spike-driven transformers [6], the state-of-the-art architecture for SNNs, which is absent from the evaluations presented in ICLR24 [4]. By focusing on up-to-date and state-of-the-art comparisons, we aim to present a comprehensive and relevant evaluation of our proposed approach.

References:

[1] Lam et al. "Precision Batching: Bitserial Decomposition for Efficient Neural Network Inference on GPUs." In PACT 2021.

[2] Yuhang et al., “Additive powers-of-two quantization: An efficient non-uniform discretization for neural networks." In ICLR 2020.

[3] Shen et al., "Are Conventional SNNs Really Efficient? A Perspective from Network Quantization." In CVPR 2024.

[4] Datta et al., "Can we get the best of both Binary Neural Networks and Spiking Neural Networks for Efficient Computer Vision?" In ICLR 2024.

[5] Chowdhury et al., “Towards ultra low latency spiking neural networks for vision and sequential tasks using temporal pruning.” In ECCV 2022.

[6] Yao et al., “Spike-driven transformer.” In NeurIPS 2023.

[7] N. Rathi et al., "DIET-SNN: A Low-Latency Spiking Neural Network With Direct Input Encoding and Leakage and Threshold Optimization." In TNNLS 2023.

[8] Horowitz, “1.1 computing’s energy problem (and what we can do about it).” In ISSCC 2014.

Q4: Several prior SNN works [3-4] have explored the interplay between the ANN activation bit-width and SNN time steps (that forms the foundation of this work), however, they have been ignored in this work.

A4: Thank you for pointing out these prior works. We appreciate the opportunity to clarify how our approach builds upon and differs from the methods discussed in [3-4]. We have now added a paragraph in Section 2.5 of our submission to better contextualize these methods relative to our work.

The authors of [4] primarily focus on binary-activation ANNs, which are equivalent to unit-timestep SNNs [5] and which we have compared to in Table 1 of our submission. In contrast, [3] introduces uniform activation quantization in SNNs to preserve higher activation precision per timestep while offloading the burden of multi-timestep processing. However, this approach compromises the multiplication-free property of SNNs. Our proposed one-hot M-LIF neuron model diverges from these approaches by employing a one-hot (power-of-two) encoding of activations, which allows only one of the $S$ spike lanes to fire per timestep. This one-hot constraint is trained and maintained throughout training—a critical distinction not present in [3-4]. Moreover, the output activations of our neuron correspond to powers-of-two and are efficiently used to update the membrane potential by modifying FP32 weight exponents using a single INT8 addition, as opposed to requiring multiplications like the 4-bit uniformly quantized activations described in [3]. Regarding evaluations, CVPR24 [3] emphasizes joint quantization of both activations and weights for distinct neural network architectures from those in our study, making direct comparisons challenging. Weight quantization, as explored in [3], is orthogonal to our work and could complement one-hot M-LIF SNNs for potential further energy optimization. While ICLR24 [4] does not demonstrate the applicability of its method on spike-driven transformer architectures [6], which is the state-of-the-art SNN architecture backbone, we were able to draw comparisons using VGG16 on ImageNet. ICLR24 [4] uses a comparable computational energy model to the one used in our study and also provides memory energy metrics and spike rates, facilitating a comparative analysis. Since our submission already compares results using a 45nm technology baseline [8], we recalculated the energy consumption metrics of VGG16 from [4] using the same 45nm values from [8]. Additionally, one of ICLR24 [4]'s key contributions is reducing timesteps down to 1 without iterative temporal pruning, as employed in ECCV22 [5]. Similarly, our method achieves unit-timestep SNN training without temporal pruning while also offering a superior overall accuracy-energy tradeoff compared to standard ANNs as shown in the table below. One-hot M-LIF SNNs achieve energy efficiency that is $33\times$ lower than ANNs, while maintaining higher accuracy (up to $> 3$) than prior unit-timestep SNN works. Furthermore, we highlight the advantage of using unit-timestep processing in this table since we achieve higher accuracy compared to [7] while consuming $2.5\times$ less total energy, the majority of which stems from the memory energy overhead of multi-timestep ($T=5$) processing.

See references in previous comments

Q2: I understand there is one-hot encoding in the spike lanes, however, this contribution alone may not be sufficient to warrant acceptance in ICLR. This is also similar to power-of-two quantization [2] (https://openreview.net/pdf?id=BkgXT24tDS), that has been extensively explored in several quantization works

A2: Thank you for referencing ICLR20 [2]. We appreciate the opportunity to clarify the differences between our proposed approach and additive power-of-two quantization as described in the cited work. In ICLR20 [2], quantization levels are represented as sums of powers-of-two, which consequently introduces additional add operations during multiplication with inputs at these quantization levels. In contrast, our proposed one-hot M-LIF SNNs ensure only single (one-hot) power-of-twos inputs/outputs, maintaining a simpler and more efficient operation. Moreover, ICLR20 [2] does not explore the application of one-hot (power-of-two) encoded input/output representations within the domain of spiking neural networks (SNNs) or how these representations can enable improved accuracy-latency and energy trade-offs.

We have acknowledged prior work on power-of-two quantization in our submission, specifically in Sections 3.2 and 2.4, where we discuss log-quantized artificial neural networks (LQ-ANNs). Our comparisons illustrate key distinctions between LQ-ANNs and one-hot M-LIF SNNs, particularly in terms of training approaches and applicability to neuromorphic tasks such as dynamic vision classification. These differences include: (1) during inference, neuron dynamics differ when the time step $T>1$ due to the extensibility of M-LIF neurons to neuromorphic tasks, and (2) during training, we leverage back-propagation through time for $T>1$ while also learning threshold and leakage parameters for static tasks.

To the best of our knowledge, this work is the first to introduce a one-hot encoding scheme in SNNs. The one-hot M-LIF neuron we propose retains the multiplication-free property characteristic of SNNs, as it efficiently modifies the exponents of FP32 weights using a single INT8 addition during membrane potential updates. Moreover, due to the one-hot constraint, the overall spiking rate (and consequently the number of additions) per layer per timestep in one-hot M-LIF SNNs is not necessarily higher than that of conventional SNNs, even with multiple spiking lanes per neuron. This allows one-hot M-LIF SNNs to extract greater learning potential within a single timestep while maintaining comparable computational complexity and energy efficiency relative to traditional SNNs. This is evident in how one-hot M-LIF spike-driven transformers in Table 1 of our manuscript can preserve accuracy better than traditional spike-driven transformers when reducing the number of timesteps from $4$ to $1$. Integrating the one-hot constraint is nontrivial, as it necessitates modifications to both the neuron model and the surrogate gradients. We have demonstrated its efficacy through extensive evaluations across various neural network architectures and datasets, including both static and dynamic tasks, as detailed in our submission.

References:

[1] Lam et al. "Precision Batching: Bitserial Decomposition for Efficient Neural Network Inference on GPUs." In PACT 2021.

[2] Yuhang et al., “Additive powers-of-two quantization: An efficient non-uniform discretization for neural networks." In ICLR 2020.

[3] Shen et al., "Are Conventional SNNs Really Efficient? A Perspective from Network Quantization." In CVPR 2024.

[4] Datta et al., "Can we get the best of both Binary Neural Networks and Spiking Neural Networks for Efficient Computer Vision?" In ICLR 2024.

[5] Chowdhury et al., “Towards ultra low latency spiking neural networks for vision and sequential tasks using temporal pruning.” In ECCV 2022.

[6] Yao et al., “Spike-driven transformer.” In NeurIPS 2023.

[7] N. Rathi et al., "DIET-SNN: A Low-Latency Spiking Neural Network With Direct Input Encoding and Leakage and Threshold Optimization." In TNNLS 2023.

[8] Horowitz, “1.1 computing’s energy problem (and what we can do about it).” In ISSCC 2014.