TTFSFormer: A TTFS-based Lossless Conversion of Spiking Transformer

Thanks for your suggestions. We would like to address your concerns and questions in the following.

Code implementation

Thank you for your question. Our code will be released with the publication.

Is there a way to quantify the minimum processing latency of a TTFSFormer neuron block?

Thank you for your question. The precision $p$ represents the precision of time in hardware implementation. While the accuracy of our converted model drops with lower precision, the problem can be solved with fine-tuning, as explored in previous works ([Stanojevic et al. 2024]).

The precision does not reflect the latency of the inference, but a shorter latency leads to a lower precision on a given hardware. However, there are still other techniques that can reduce the hardware latency. In order to further reduce the hardware latency, we can follow the method proposed in [Park et al. 2020], in which the emitting stage and receiving stage can be overlapped in order to reduce the latency. This requires a balance between the conversion loss and latency. Since our work mainly focuses on a lossless conversion method of Transformer architecture, we leave the topic for future works.

How are the OPs counted for a given transformer architecture?

Thank you for your question. We will include a more detailed analysis in future versions of our paper.

We briefly sketch the computing process in the following. Take one block in ViT as an example.

• In the converted SNN, we can divide the operations into two categories: non-linear operations and linear operations. Linear operations take place in neurons with dynamics described in Corollary 4.4 and 4.6, which is exactly what most previous works on TTFS-based SNN use. Non-linear operations are new in our work, and the neurons may require more energy, depending on the hardware implementation.
• In the original ANN, the energy consumption is measured by the number of multiplication and addition operations.

All operations are shown below (d = 384 or 768 or 1024 in ViT-S/B/L respectively). We categorize layers according to their type.

(The number of operations in each SNN layer is analyzed in section 4.3 and appendix C.)

By summing up operations in each type of layer, we can finally get the estimated OPs in ANN and SNN. Thus, we have $197 \cdot (1206d+1578)$ non-linear operations and $197 \cdot 11d^{2}$ linear operations in one block.

Now we can estimate the energy efficiency by (we divide the energy by 197 simultaneously)

$$\eta = \frac{\text{Energy of SNN}}{\text{Energy of ANN}} = \frac{(1206d+1578)E_{nl} + 11d^{2}E_{l}}{(11d^{2}+400d+197)E_\text{MAC}} \approx \frac{1206 \cdot E_{nl} + 11d \cdot E_{l}}{(11d + 400)E_\text{MAC}}$$

where $E_\text{MAC}$ is the energy consumption of one multiply-add operation in ANN, $E_{l}, E_{nl}$ is the energy consumption of one linear and non-linear operation in SNN.

We use the settings in [Horowitz, 2014], namely $E_{l} = 0.9 \text{pJ},E_\text{MAC}=4.6\text{pJ}$. We assume that $E_{nl} = kE_{l}$, where $k$ is a constant depending on the hardware implementation.

Take ViT-L for example, by letting $d=1024$, we have

$$\eta = 0.189 + 0.020k$$

In our paper, we assume that $k=1$, i.e. the non-linear neuron has the same cost with the linear neuron. We think that a reasonable estimation would be $E_{l} < E_{nl} < E_\text{MAC}$, for example $k=2$ or $k=3$. Although $k$ may vary according to the hardware implementation, we can conclude that the energy efficiency is between 20%-30%.

[Stanojevic et al. 2024] Stanojevic, A., Woźniak, S., Bellec, G. et al. High-performance deep spiking neural networks with 0.3 spikes per neuron. Nat Commun 15, 6793 (2024).

[Park et al. 2020] S. Park, S. Kim, B. Na and S. Yoon, "T2FSNN: Deep Spiking Neural Networks with Time-to-first-spike Coding," 2020 57th ACM/IEEE Design Automation Conference (DAC), San Francisco, CA, USA, 2020, pp. 1-6

Thank you for your detailed feedback. We are encouraged that you found our work novel with good results. We would like to address your concerns and questions in the following.

1. Table 1 lacks comparisons to very recent TTFS-based methods or advanced rate-coding SNN Transformers.

Thanks for your suggestion. We add the comparision with more recent works as follows.

In Table 2 of our paper and the table above, direct training refers to training SNN directly through the surrogate gradient method. In Table 2 of our paper, work not marked with "TTFS-based" is rate-based method by default.

[Wang et al. 2025] Ziqing Wang and Yuetong Fang and Jiahang Cao and Hongwei Ren and Renjing Xu, "Adaptive Calibration: A Unified Conversion Framework of Spiking Neural Network", AAAI 2025.

[Yao et al. 2025] Man Yao, et al. "Scaling spike-driven transformer with efficient spike firing approximation training." IEEE Transactions on Pattern Analysis and Machine Intelligence (2025).

[Zhou et al. 2024] Chenlin Zhou et al. "QKFormer: Hierarchical Spiking Transformer using Q-K Attention", NIPS 2024.

[Yao et al. 2024] Man Yao, et al. "Spike-driven Transformer V2: Meta Spiking Neural Network Architecture Inspiring the Design of Next-generation Neuromorphic Chips.", ICLR 2024.

2. Discuss hardware implementation challenges and latency trade-offs of the proposed model could enhance the future work.

Hardware Implementation Challenge

We think that potential challenges in hardware implementation include:

• Design special hardware that is compatible with the non-linear neurons proposed in our method. With different dynamics compared with traditional LIF models, the hardware needs a special design in order to fully demonstrate the energy efficiency of our proposed methods.
• Deal with hardware-related accuracy loss. Although we have proved that our conversion method is theoretically lossless, there will inevitably exist hardware-specific loss, including noise in membrane potential and time bias of TTFS spikes. We have discussed the robustness of our proposed method in Section 5.3, and more analysis is needed for hardware implementation. Fine-tuning is probably needed if the hardware loss is too large.

Latency Trade-offs

In our proposed method, there are two separate stages for one layer, namely, the emitting stage and the receiving stage. As proposed in [Park et al. 2020], the two stages can be overlapped in order to reduce the latency, requiring a balance between the conversion loss and latency reduction. Since our work mainly focuses on a lossless conversion method of Transformer architecture, we leave the topic for future works.

We will add more discussion in the revised version.

3. The TTFS could provide advantage in the aspect of low firing rates, hence there could supplement the firing rate comparison and analysis.

Thank you for your inspiring question. Compared with previous work on TTFS, each non-linear neuron emits exactly one spike every forward pass. The firing rate is relatively lower than the rate-based method since multiple time steps is needed in rate-based SNN, leading to multiple spikes emitted in one neuron. However, the firing rate is higher than previous work on TTFS-based converted CNN, since we cannot simply ignore those negative values in the network with non-ReLU activation functions and attention mechanisms.

The firing rate can be further cut down. For example, in activation function SiLU, we can regard $\mathrm{SiLU}(x) \approx 0$ for $x \le -16$ and simply emit no spike, since $\mathrm{SiLU}(-16) \approx -1.8 \times 10^{-6}$ is close to 0. In this way, we can still keep the information carried by values around zero while enhancing energy efficiency.

[Park et al. 2020] S. Park, S. Kim, B. Na and S. Yoon, "T2FSNN: Deep Spiking Neural Networks with Time-to-First-Spike Coding," 2020 57th ACM/IEEE Design Automation Conference (DAC), San Francisco, CA, USA, 2020, pp. 1-6

Thank you for your thoughtful comments. We would like to address your concerns in the following.

Hardware Implementation

Thanks for your question. Since rate-based SNN is currently the most popular coding method, existing neuromorphic chips are specially designed for rate-based algorithms. With existing hardware circuits, one possible workaround is to store the mapping of non-linear functions in hardware storage, and simulate the change of potential. This is easy to implement since the non-linear function is shared in one layer. We hope that there will be neuromorphic chips suitable for TTFS-based SNN models, which will better demonstrate the energy efficiency of temporal coding methods. We will add more discussion in the revised manuscript.

Scalability Concerns

Different from other tasks such as NLP, the size of vision models typically ranges from 80M (such as ViT-B) to 1000M. In order to better demonstrate the scalability of our proposed method, we test our method on EVA-G (Giant) with 1.01B parameters in total, which is one of the largest popular pre-trained vision transformers. The experimentable result is shown in the table below.

The result shows that our conversion method is still virtually lossless when converting a large vision model, indicating that our method has excellent scalability in vision task.

Limited Novelty in SNN Design

We agree that the idea of time-to-first-spike (TTFS) coding has been put forward and explored in previous works. Many SNN algorithms, either direct training or conversion, have created convolutional SNN successfully with much lower energy consumption and comparable performance to CNN. However, we would like to clarify that our work is the first to focus on the transformer model with TTFS coding.

Our work mainly focuses on analyzing and solving the challenges encountered in introducing transformer architecture into TTFS-based SNN, which has not been addressed in previous works.

• Challenge 1: Transformer architecture involves plenty of non-linear operations, such as non-ReLU activations, attention, and LayerNorm. However, previous TTFS neuron dynamics are only able to represent piecewise linear functions (Section 3.3).
• Our Solution 1: We proposed a non-linear neuron model, which introduces non-linearity in SNN. We have theoretically proved that our neuron model has a strong representation ability (Section 4.2). With the proposed neuron model, we can construct Transformer-specific components (in Section 4.3).
• Challenge 2: Previous TTFS neurons have a small representation range.
• Our Solution 2: We expand the representation range of TTFS neurons depending on the actual distribution of the input and output values.
• Challenge 3: Accuracy of converted models.
• Our Solution 3: We have theoretically proved that our conversion method is lossless. Moreover, we conduct experiments on various vision transformer models (ViT, EVA) with different sizes (from 80M to 1B), showing that our method can achieve SOTA performance in vision tasks.

Insufficient Comparison

Thanks for your suggestion. We add the comparison with more recent works as follows.

In Table 2 of our paper and the table above, direct training refers to training SNN directly through the surrogate gradient method. In Table 2 of our paper, work not marked with "TTFS-based" is rate-based method by default.

[Wang et al. 2025] Ziqing Wang and Yuetong Fang and Jiahang Cao and Hongwei Ren and Renjing Xu, "Adaptive Calibration: A Unified Conversion Framework of Spiking Neural Network", AAAI 2025.

[Yao et al. 2025] Man Yao, et al. "Scaling spike-driven transformer with efficient spike firing approximation training." IEEE Transactions on Pattern Analysis and Machine Intelligence (2025).

[Zhou et al. 2024] Chenlin Zhou et al. "QKFormer: Hierarchical Spiking Transformer using Q-K Attention", NIPS 2024.

[Yao et al. 2024] Man Yao, et al. "Spike-driven Transformer V2: Meta Spiking Neural Network Architecture Inspiring the Design of Next-generation Neuromorphic Chips.", ICLR 2024.

Thank you for your positive and thoughtful comments. We are encouraged that you find our idea impressive. We are glad you agree that our method achieves SOTA performance with low energy consumption. We would like to address your concerns and questions in the following.

1. Please clarify how to compute the energy consumption in Table 2.

Thanks for your question. Our energy estimation follows the methodology proposed by [Nitin Rathi and Kaushik Roy, 2020]. For clarity, we analyze operations in a ViT block, categorizing them into linear (standard TTFS neuron dynamics, as in prior work) and non-linear (novel to our method, with hardware-dependent energy costs). Below is the list of operations (where d=384 or 768 or 1024 for ViT-S/B/L respectively):

(The number of operations in each SNN layer is analyzed in section 4.3 and appendix C.)

Now we can estimate the energy efficiency:

$$\eta = \frac{\text{Energy of SNN}}{\text{Energy of ANN}} = \frac{(1206d+1578)E_{nl} + 11d^{2}E_{l}}{(11d^{2}+400d+197)E_\text{MAC}} \approx \frac{1206 \cdot E_{nl} + 11d \cdot E_{l}}{(11d + 400)E_\text{MAC}}$$

where $E_\text{MAC}$ is the energy consumption of one multiply-add operation in ANN, $E_{l}, E_{nl}$ is the energy consumption of one linear and non-linear operation in SNN.

We use the settings in [Horowitz, 2014], namely $E_{l} = 0.9 \text{pJ},E_\text{MAC}=4.6\text{pJ}$. We assume that $E_{nl} = kE_{l}$, where $k$ is a constant depending on the hardware implementation.

Take ViT-L for example, by letting $d=1024$, we have $\eta = 0.189 + 0.020k$.

In our paper, we assume that $k=1$, i.e. the non-linear neuron has the same cost with the linear neuron. We think that a reasonable estimation would be $E_{l} < E_{nl} < E_\text{MAC}$, for example $k=2$ or $k=3$. Although $k$ may vary according to the hardware implementation, we can conclude that the energy efficiency is between 20%-30%.

2. Could the proposed method be used to other models? Like CNN and ResNet.

Thanks for your question. Our work generalizes prior TTFS-based methods from CNNs (consisting of linear, convolutional layers and ReLU) to Transformers. For CNN/ResNet architectures, our model reduces to the linear neuron dynamics described in Corollary 4.4 and 4.6, which is almost exactly what existing TTFS approaches (e.g. [Stanojevic et al. 2024]) are. While our method is compatible with CNN architectures, this would not introduce novel improvements for CNNs.

3. Whether the proposed method can be generalized to the language task? other visual tasks?

Thanks for your inspiring discussion. While our experiments focus on vision tasks, the method is theoretically compatible with standard Transformer architectures in NLP. However, the generalization ability requires further experiments. For example, LLMs are typically larger than vision models, which poses challenges to the scalability of our method. We view this as a promising direction that desires future work. Besides, our method can be applied to regression-based vision tasks like object detection. We will add more discussion.

4. Computational cost of the generalized TTFS neuron

The energy consumption of our non-linear neuron will be similar to or slightly higher than that of the LIF neuron. LIF neurons can be regarded as a special case of non-linear model where the potential change $\eta,\psi$ are exponential. When the potential change is not exponential, special hardware design is required, which is expected to have a similar energy cost with LIF neurons.

[Nitin Rathi and Kaushik Roy, 2020] Nitin Rathi and Kaushik Roy. 2020. Diet-SNN: Direct Input Encoding with Leakage and Threshold Optimization in Deep Spiking Neural Networks.

[Horowitz, 2014] M. Horowitz, "1.1 Computing's energy problem (and what we can do about it)," 2014 IEEE International Solid-State Circuits Conference Digest of Technical Papers (ISSCC), San Francisco, CA, USA, pp. 10-14

[Stanojevic et al. 2024] Stanojevic, A., Woźniak, S., Bellec, G. et al. High-performance deep spiking neural networks with 0.3 spikes per neuron. Nat Commun 15, 6793 (2024).