MI-TRQR: Mutual Information-Based Temporal Redundancy Quantification and Reduction for Energy-Efficient Spiking Neural Networks

Thank you very much for taking the time to review our paper. Our responses to your questions are as follows:

1. Question: Theoretical analysis. Although the authors claim that mutual information is an effective way to evaluate the similarity between discrete variables, there is a lack of theoretical analysis regarding the effectiveness of binary spikes. Does the limited range of representations of binary spikes affect the precision of mutual information?

Response: Thanks for the meticulous comment.

(1) Mutual information (MI) is a widely used non-linear metric for quantifying statistical dependence between discrete variables. In the context of binary spike features, where each element is assigned a value of 0 (no spike) or 1 (spike), MI remains theoretically valid and interpretable.

(2) The state space of binary variables ({0,1}) does not limit the theoretical precision of MI. As established in classical information theory [1], MI measures the reduction in uncertainty of one variable given the knowledge of another. For binary variables, MI is bounded between 0 and 1, with 1 indicating identical variables and 0 indicating completely independent variables. Empirical evidence also supports the utility of MI: for example, MI increases monotonically with similarity intensity, even when applied to discrete or event-based representations [2]. In a word, the limited range of representations of binary spikes does not affect the precision of MI.

2. Question: Complexity and energy efficiency. Reducing the number of 1-value spikes in the SNN can reduce deployment power consumption by avoiding addition operations caused by those spikes. However, computing mutual information requires a large number of multiplications and logarithmic operations (Equations 1 and 2), which consume more power than addition. In fact, this increases the overall inference power and complexity rather than decreasing them. Therefore, the authors should not rely solely on the number of spikes to calculate the theoretical power consumption.

Response: Yes. You are right. Computing mutual information requires a large number of multiplications and logarithmic operations. Nevertheless, as stated in line 525, our module is applied solely during training and has no impact on the inference stage. Therefore, it is reasonable to rely solely on the number of spikes to calculate the theoretical energy consumption. We will include the related statement (our module is only used during training) in the main text.

3. Question: Method rationality. The authors retained all spikes from the initial time step and used them as references to calculate probabilities for subsequent time steps. However, these calculations did not consider interactions between other time steps. Furthermore, recent work [3.4] has shown that the performance of SNNs degrades dramatically when relying only on early time steps. Does this suggest that the limited amount of information in the first time step cannot be used as a reference benchmark?

Response: Thanks for your insightful and valuable thinking.

(1) The interactions between other time steps.

The temporal information concentration (TIC) phenomenon indicates that, as training goes on, the amount of information at the first timestep increases while that of the other timesteps decreases. Therefore, we argue that especially in the later stages of training, the first time step carries the most critical information and can be used as a reference, whereas the others become negligible and unsuitable for reference. Accordingly, we measure the mutual information between the features at each subsequent timestep and those at the first timestep. Even when mutual information is measured between features from other timesteps, the resulting values are of limited significance, since both features convey little information on their own.

(2) The amount of information in the first time step.

We study these two works [3.4] and find the related demonstration as follows:

We view this averaging operation as an ensemble strategy: averaging the different outputs of multiple models improves overall performance. [3]

Inspired by ensemble learning, viewing the mean output of SNNs as an ensemble aggregated through voting over time, it becomes apparent that the overall outcome across these temporal dimensions tends to improve as the accuracy at each individual timestep increases. [4]

We also agree that averaging the different outputs of multiple models improves overall performance. We argue that this is not in conflict with the TIC phenomenon [5], which demonstrates that information gradually concentrates on the first timestep as training goes on. In other words, the TIC phenomenon means that the improvement extent will gradually decrease as training goes on. Moreover, to avoid misunderstanding, we will modify the description on line 26 in the final version as follows:

SNNs rely on sequential timesteps to transmit temporal information, and their outputs are typically averaged across time to improve accuracy [3,4]. The temporally shared weights induce spatio-temporal invariance, introducing substantial feature redundancy and resulting in a high spiking firing rate, as shown in Fig. 1(a).

4. Question: Method effectiveness. The authors point out that removing spikes results in an imbalanced distribution across time steps. Therefore, they reweight the weights for each time step. However, the authors calculated the weights using firing rates only and did not adjust them for the unbalanced distribution. Even more importantly, no ablation studies have verified the effectiveness of this reweighting.

Response: Thanks for your careful review. We conduct the ablation study to verify the effectiveness of this reweighting.

We can observe that the reweighting is effective. We will include the ablation study in the final version.

5. Question: Insufficient experimental comparisons. Although the related work is described, the authors do not compare the proposed method with other lightweighting or redundancy reduction methods in the experimental section. This makes it difficult to verify whether the proposed method has any significant advantages. It is recommended that the authors add a detailed comparison with other redundancy reduction methods such as ASA [6] and RM-SNN [7].

Response: I have compared our method to ASA [6] on Gait. For CIFAR10-DVS, RM-SNN obtained 82.9% with $T$=10. Our method achieved 86.5%. We will include the comparison in the final version.

6. Question: Insignificant performance gains. The performance improvement on the time-series forecasting task is too minimal to determine whether the method is effective. More experimental evidence is recommended. Additionally, are the results reported in the paper the best results obtained from multiple experiments, such as CIFAR10-DVS? If so, please provide the average results.

Response: Thanks for your careful consideration.

(1) Although the performance improvement is not overwhelming, it is sufficient to demonstrate the effectiveness of our method to learn more compact and powerful representations. Because our MI-TRQR not only increases the accuracy but also reduces the energy consumption compared to multiple SNN baselines for various tasks. We provide much evidence in various ways.

(2) We follow your advice and run three experiments. The result on CIFAR10-DVS with T=4 is 83.9$\pm$0.08.

References

[1] Thomas M. Cover, and Joy A. Thomas. Elements of information theory {(2.} ed. 2006

[2] Thomas Kreuz, Florian Mormann, Ralph G. Andrzejak, Alexander Kraskov, Klaus Lehnertz, and Peter Grassberger. Measuring synchronization in coupled model systems: A comparison of different approaches. Physica D, Nonlinear Phenomena, 2007, 225(1): 29-42.

[3] Yongqi Ding, Lin Zuo, Mengmeng Jing, Pei He, and Hanpu Deng. Rethinking Spiking Neural Networks from an Ensemble Learning Perspective. ICLR. 2025.

[4] Chengting Yu, Xiaochen Zhao, Lei Liu, Shu Yang, Gaoang Wang, Erping Li, and Aili Wang. Efficient Logit-based Knowledge Distillation of Deep Spiking Neural Networks for Full-Range Timestep Deployment. ICML. 2025.

[5] Youngeun Kim, Yuhang Li, Hyoungseob Park, Yeshwanth Venkatesha, Anna Hambitzer, and Priyadarshini Panda. Exploring temporal information dynamics in spiking neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 8308–8316, 2023.

[6] Man Yao, Jiakui Hu, Guangshe Zhao, Yaoyuan Wang, Ziyang Zhang, Bo Xu, and Guoqi Li. Inherent Redundancy in Spiking Neural Networks. ICCV. 2023.

[7] Man Yao, Hengyu Zhang, Guangshe Zhao, Xiyu Zhang, Dingheng Wang, Gang Cao, and Guoqi Li. Sparser spiking activity can be better: Feature Refine-and-Mask spiking neural network for event-based visual recognition. 2023.

Thank you very much for taking the time to review our paper. Our responses to your questions are as follows:

1. Question: The ablation study shows that applying the MI-TRQR module after the last convolutional layer yields the best results. Could the authors provide further insight into this observation? This result is somewhat puzzling, as temporal redundancy likely exists in intermediate layers as well. In addition, have the authors considered applying MI-TRQR at multiple stages throughout the network?

Response: Thanks for your consideration. We address this concern in the following aspects:

(1) Why do we apply the MI-TRQR module after the last convolutional layer?

Prior studies [1, 2] show the final convolutional layer features are most critical for correct classification, while earlier layers mainly learn reusable general representations and deeper layers become highly specific. Placing MI-TRQR after the last layer therefore helps the classifier directly filter out redundant spikes, and this decision can in turn dynamically guide the removal of redundancy in shallower layers. Its rigorous analysis and derivation are provided in Appendix B, a similar statement in line 160.

(2) Why we did not apply MI-TRQR at multiple stages?

Redundancies at shallow and deep layers are not equivalent. Specifically, the redundancy identified in shallow layers may be useful in deep layers. Features that are considered useful in shallow layers may be useless in deep layers.

(3) The computation of probability $p_{t,h,w}$ (Eq. 5) is spatiotemporally dependent, with a computational complexity of $\mathcal{O}(T)$ for global redundancy and $\mathcal{O}(T×H×W)$ for local redundancy. Therefore, inserting the module into earlier layers leads to higher computational cost. We provide the corresponding time of training an epoch based on Tab. 5. Due to time constraints, we only obtain the results of the module inserted after layers 6 and 8.

We can see that adding the module to earlier layers introduces substantial computational cost, yet yields no gains in accuracy or firing rate reduction.

2. Question: Regarding the reported power consumption: how is the power measured? Is it obtained from real hardware or through simulation? If the latter, please clarify the simulation method and include the formulation used to estimate power consumption.

Response: The power consumption is obtained through simulation. The simulation method follows prior works [3,4], and the estimation formulation is provided in Eq. 11 (page 15). We show it as follows:

$$E_{net}=E_{MAC} \cdot FL_{Conv}^{1} + E_{AC} \cdot T \cdot (\sum_{n=2}^{N}FL_{Conv}^{n}\cdot fr_{Conv}^{n}+ \sum_{m=1}^{M}FL_{FC}^{m}\cdot fr_{FC}^{m})$$

where $FL_{Conv}^{1}$ is the FLOPs of first convolution layer, $FL_{Conv}^{n}$ and $FL_{FC}^{m}$ are the FLOPs of the $n$-th convolution layer and the $m$-th FC layer, $fr_{Conv}^{n}$, and $fr_{FC}^{m}$ are the firing rate of the $n$-th convolution layer and the $m$-th FC layer, respectively. $N$ and $M$ are the total number of convolution and FC layers. Following prior works [5,6], $E_{MAC}$=4.6pJ, $E_{AC}$=0.9pJ.

References

[1] Jie Hu, Li Shen, and Gang Sun, Squeeze-and-excitation networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 7132–7141, 2018.

[2] Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, and Antonio Torralba. Learning deep features for discriminative localization. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 2921–2929, 2016.

[3] Hangchi Shen, Qian Zheng, Huamin Wang, and Gang Pan. Rethinking the membrane dynamics and optimization objectives of spiking neural networks. Advances in Neural Information Processing Systems, 37:92697–92720, 2024.

[4] Xuerui Qiu, Rui-Jie Zhu, Yuhong Chou, Zhaorui Wang, Liang-jian Deng, and Guoqi Li. Gated attention coding for training high-performance and efficient spiking neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 601–610, 2024.

[5] Mark Horowitz. 1.1 computing’s energy problem (and what we can do about it). In 2014 IEEE International Solid-State Circuits Conference Digest of Technical Papers (ISSCC), pages 10–14. IEEE, 2014.

[6] Bojian Yin, Federico Corradi, and Sander M Bohté. Accurate and efficient time-domain classification with adaptive spiking recurrent neural networks. Nature Machine Intelligence, 3(10):905–913, 2021.

Thank you very much for taking the time to review our paper. Our responses to your questions are as follows:

1. Question: The addition of the MI-TRQR module, while reducing energy consumption leads to additional parameters, but the number of parameters (memory) could be a bottleneck depending on the application considered.

Response: Our MI-TRQR requires extra computational and memory overhead but no additional parameters. Our MI-TRQR is only applied during training, which is stated in line 525. At the inference stage, our method behaves identically to the baseline model. Therefore, our method does not pose a memory bottleneck in various applications. We will include the related statement (our module is only used during training) in the main text.

2. Question: I find it unclear how the weight recalibration takes place. It appears that the input spikes are still projected through some sort of residual connections (see Fig. 2, dashed red line). Equations 8 and 9 however appear to specify otherwise, as the spiking rate (which should be, I reckon the one after spike removal) of the network is used to calculate α.

Response:

(1) Yes. Your understanding is correct. The weight $a_t$ is computed based on the spiking firing rate of the network at the timestep $t$. Specifically, we count the number of spikes and total elements after each spiking neuron at every timestep and use this to derive the firing rate used for weight recalibration.

(2) We apologize for the discontinuity of the dashed red line in Fig. 2. It should be connected to the same line in the upper left corner, indicating the network spiking firing rate. The break was introduced to avoid visual clutter in the middle part.

3. Question: While the MI-TRQR module allows for the removal of input spikes, the module in itself is not spike-based as it calculates and optimizes continuous quantities, which I find unfortunate for the deployment of fully spiking-based neural network processes.

Response: Your question is very insightful. Our MI-TRQR calculates continuous quantities. Since current neuromorphic chips still support operations on continuous values. The chips are energy-efficient as long as the outputs of spiking neurons are binary (0 or 1). Therefore, our module remains compatible with the deployment of fully spiking-based neural network processes.

4. Question: I would kindly ask the authors to please clarify further how the weight recalibration happens in the network.

Response: During training, at the $l$-th layer of spiking neuron, we count the number of spikes $N_{l,t}^{s}$ and total number of elements $N^e_{l,t}$ at each timestep. These values are used to calculate the timestep-wise firing rate $fr^{net}_t$:

$$fr_{t}^{net} = \frac{\sum_{l=1}^{L} N_{l,t}^{s}}{\sum_{l=1}^{L} N_{l,t}^{e}}$$

where $t$ indicates the timestep. $L$ is the total number of spiking neuron layers. $fr_{t}^{net}$ is then used to calculate the weight in Eq. 8. We will provide a detailed demonstration in the final version.

5. Question: A relevant body of work which has not been discussed in the paper is the one of predictive coding [1, 2]. In predictive coding, pixel-wise redundant spikes (and internal layer spikes) are removed, usually following some sort of minimization of the so-called free-energy [1]. I believe that the authors should at the very least cite this body of work and discuss differences between their work and some of the spiking counterparts of predictive coding [2], as both types of models aim at learning efficient visual representations through the removal of redundant spikes.

Response: Thank you very much for providing these excellent works. They indeed provide additional theoretical support for spike removal in our MI-TRQR. We will cite these references and revise the motivation (line 51) as follows in the final version:

Previous studies analyzing neural recordings from various monkey brain regions have shown that spike features are significantly less redundant with activity-dependent depression. Predictive coding has also demonstrated the ability to learn efficient visual representations by removing pixel-wise redundant spikes [1,2]. Inspired by these works, we propose a parameter-free, plug-and-play MI-based Temporal Redundancy Quantification and Reduction (MI-TRQR) module, which is seamlessly integrated into the SNNs.

6. Question: I would kindly ask the authors to analyze the weaknesses/limitations of their work.

Response: Thank you for your suggestion. We will include a limitation section in the final version.

Although our method does not introduce extra cost during inference, it requires computing mutual information between spike features during training, resulting in increased computational cost and longer training time.

7. Question: I encourage the authors to modify the formatting of Sec. 4.1, as the text is not properly justified.

Response: Thank you for your suggestion. Due to space constraints, we arrange the text of Sec. 4.1 and Tab. 3 side by side. We will revise the formatting and improve layout quality in the final version.

References

[1] Bogacz, Rafal. "A tutorial on the free-energy framework for modelling perception and learning." Journal of mathematical psychology 76 (2017): 198-211.

[2] N'dri, Antony W., et al. "Predictive Coding with Spiking Neural Networks: a Survey." arXiv preprint arXiv:2409.05386 (2024).

Thank you very much for taking the time to review our paper. Our responses to your questions are as follows:

1. Reliability of the mutual information measuring technique

1.1. Question: Details of mutual information.

Response:

(1). We provide the equation of mutual information (MI) in Appendix A (line 452).

(2). The MI calculation is implemented using the TorchMetrics package (line 525). The implementation code is:

MI_value = torchmetrics.functional.clustering.mutual_info_score(S1, S2)

where S1 and S2 are two spiking features. $S1,S2\in\lbrace0,1\rbrace^{C×H×W}$ and $S1,S2\in\lbrace 0,1\rbrace ^C$ when computing global and local redundancy. MI_value is a scalar, indicating the MI between $S1$ and $S2$.

(3) It does need many samples for the computation of MI. The MI of two random variables is a measure of the mutual dependence between the two variables. Therefore, the calculation of MI only needs two spike features.

1.2. Question: Impacts of MI-TRQR for both event-driven and static datasets.

Response: The reviewer points out the similar impacts for both event-driven and static datasets. We believe the reviewer is referring to the reduction in energy consumption on CIFAR10-DVS (37.50% with T=4) and ImageNet (24.82% on ResNet50). We argue that the CIFAR10-DVS is a small dataset (10,000 samples in 10 classes), while ImageNet is a large-scale dataset (about 1.3 million images in 1,000 classes). Therefore, they are not directly comparable. We also show the impact on CIFAR10 (60,000 samples in 10 classes) in Appendix C.2. MI-TRQR reduces energy consumption by 73.48%. Therefore, our MI-TRQR achieves the expected results, in that it is much more successful at reducing spikes for data with more redundancy (static duplicated images).

1.3. Question: The computational cost of calculating mutual information.

Response: We show the time of training an epoch using different methods with a single NVIDIA RTX 3090 GPU:

2. Ablation of the role of the spike normalization

2.1. Question: The same ratios without masks to random spikes or to all spikes

Response: Thanks for your profound thinking.

(1) The mask is one concrete strategy to remove redundant spikes; other mechanisms could also be used. The reason why we use the mask is that matrix calculation is more efficient in current machines.

(2) When only global redundancy is used to remove redundant spikes, the removal within $S_t\in\lbrace0,1\rbrace^{C×H×W}$ at timestep $t$ is essentially random.

(3) We apply the same ratios to all spikes. Specifically, after obtaining the mask with $p_{t,h,w}$ (Eq. 5), we shuffle all elements to generate a random mask for spike removal.

We observe that both the MI-based ratio and the removed spikes are crucial.

2.2 Question: Static penalty of the number of spikes.

Response: We use different static penalty factors:

Static penalties are indeed effective at reducing redundant spikes. Notably, MI-TRQR achieves better results on accuracy and firing rate.

3. Question: Do MI based pruning? Swap out MI with simpler techniques (perhaps even linear). Would spiking rate still have gone down? The ablations are done only against the whole network, without any spike pruning?

Response:

(1) Spike removal and pruning serve distinct purposes: pruning discards low-impact weights, whereas our method removes redundant spikes from features. This distinction allows the two to be applied together in a complementary manner.

(2) We swap out MI with simpler techniques, such as Euclidean similarity (via 1/(1+Euclidean distance)), Pearson correlation, and Cosine similarity.

We can see that the spiking rate still goes down using simpler techniques. Notably, MI-TRQR performs better.

(3) Ablation with spike pruning.

We multiply $p_{t,h,w}$ (Eq. 5) by an additional factor $a$, and the results of different factors are:

We can see that factor $a$ affects the balance between accuracy and firing rate.

4. Question: Lesser redundant spike features? Lesser correlated, perhaps even linearly?

Response:

(1) Yes. The resulting network actually has fewer redundant spike features. We show the MI for the original and new networks on CIFAR10-DVS and ImageNet in Fig. 1 and Fig. 4, respectively. Specifically, the spike features of the original network at four timesteps on CIFAR10-DVS are placed in Fig. 1(a), and their MI matrix is placed in Fig. 1(b). The spike features of the new network at four timesteps on CIFAR10-DVS are placed in Fig. 1(c), and the measured MI is placed in Fig. 1(d). We show the results on ImageNet using the same layout in Fig. 4. As shown in Fig. 1, MI-TRQR reduces the MI on CIFAR10-DVS by about 24% (from 0.1507 to 0.1134) and on ImageNet by about 79% (from 0.5837 to 0.1081). The related statements about Fig. 1 are in lines 28, 50, and 62.

(2) Using the results in Question 3(3), we compute the MI of spike features (Fig. 1) using different $a$.

$a$=0.5:

$a$=2:

We can see that it is not linear.

5. Question: Why Bernoulli? Why not threshold against a value?

Response:

(1) Bernoulli-based masks admit a clearer theoretical interpretation. In contrast, a threshold lacks such grounding.

(2) Bernoulli-based masks remove redundant spikes in a data-dependent manner and avoid introducing extra hyperparameters. The specific threshold is a hyperparameter that requires extensive studies to find the optimal configuration on different datasets.

(3) To further verify the effectiveness of our method, we define a global redundancy-based threshold for pixel-wise spike removal. Specifically, for the feature $S_t\in\lbrace0,1\rbrace^{C\times H\times W} (t>1)$, we compute the global redundancy $p_t^g$ (Eq. 3) and the local redundancy $p_{t,h,w}^l$ (Eq. 4). If $p_{t,h,w}^l>p_t^g$, the spike feature $S_{t,h,w}\in\lbrace0,1\rbrace^C$ is set to 0.

These results demonstrate that our method with Bernoulli performs better.

6. Question: The placement experiment (line 208-211). Why not across all layers?

Response: Thanks for your consideration.

(1) Why is our MI-TRQR placed after only the last layer?

Prior studies [1, 2] show the final convolutional layer features are most critical for correct classification, while earlier layers mainly learn reusable general representations and deeper layers become highly specific. Placing MI-TRQR after the last layer therefore helps the classifier directly filter out redundant spikes, and this decision can in turn dynamically guide the removal of redundancy in shallower layers. Its rigorous analysis and derivation are provided in Appendix B, a similar statement in line 160.

(2) Why not across all layers?

First, redundancies at shallow and deep layers are not equivalent. Specifically, the redundancy identified in shallow layers may be useful in deep layers. Features that are considered useful in shallow layers may be useless in deep layers.

(3) The computation of probability $p_{t,h,w}$ (Eq. 5) is spatiotemporally dependent, with a computational complexity of $\mathcal{O}(T)$ for global redundancy and $\mathcal{O}(T×H×W)$ for local redundancy. Therefore, inserting the module into earlier layers leads to higher computational cost. We provide the corresponding time of training an epoch based on Tab. 5. Due to time constraints, we only obtain the results of the module inserted after layers 6 and 8.

We can see that adding the module to earlier layers introduces substantial computational cost, yet yields no gains in accuracy or firing rate reduction.

7. Question: Algorithm 1 is not referenced.

Response: Algorithm 1 is referenced at line 159.

8. Question: Tables should occur on the same page as they are referenced, ideally.

Response: Thanks for your advice. We will try our best to improve the layout.

References

[1] Squeeze-and-excitation networks. CVPR 2018

[2] Learning deep features for discriminative localization. CVPR 2016

Thanks very much for your question. We confirm that the mutual information (MI) calculation is correct, and we clarify this from three perspectives:

(1) MI is defined and analyzed by Claude Shannon [1] and termed later by Robert Fano [2]. MI measures the statistical dependency between two discrete variables. Its computation only requires the inputs to be discrete variables and does not assume that they must represent cluster assignments. MI has been widely applied in various domains [3, 4], including in SNNs to quantify the information content in spike features [5, 6].

(2) In our case, MI is particularly suitable for quantifying redundancy between binary spiking vectors (0/1 sequences), which are a form of discrete variables. As discussed in Response (2) to Question 3, MI is more powerful than alternative similarity metrics.

(3) We compute MI using both the TorchMetrics and NPEET packages, and obtain consistent results. This further confirms the correctness of our implementation.

---

References

[1] Claude Elwood Shannon. A mathematical theory of communication. The Bell System Technical Journal, 27(3):379–423, 1948.

[2] Kreer, J. G. (1957). "A question of terminology". IRE Transactions on Information Theory. 3 (3): 208. doi:10.1109/TIT.1957.1057418

[3] Vergara J R, Estévez P A. A review of feature selection methods based on mutual information. Neural computing and applications, 24(1): 175-186, 2014.

[4] Batina L, Gierlichs B, Prouff E, et al. Mutual information analysis: a comprehensive study. Journal of Cryptology, 24(2): 269-291,2011.

[5] Yuhan Zhang, Xiaode Liu, Yuanpei Chen, Weihang Peng, Yufei Guo, Xuhui Huang, and Zhe Ma. Enhancing representation of spiking neural networks via similarity-sensitive contrastive learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 16926–16934, 2024.

[6] Shikuang Deng, Yuhang Wu, Kangrui Du, and Shi Gu. Spiking token mixer: An event-driven friendly former structure for spiking neural networks. In Advances in Neural Information Processing Systems, volume 37, pages 128825–128846, 2024.