S-TLLR: STDP-inspired Temporal Local Learning Rule for Spiking Neural Networks

Answer 1: Thank you for your suggestion. In our revised version, we have incorporated the results for the N-Caltech101 dataset. Regrettably, integrating the Ncars dataset into our current framework wasn't feasible at this moment. Please, note that in our initial version, we already included evaluations beyond event-based vision classification tasks. Specifically, we incorporated assessments in event-based audio recognition and optical flow domains. These evaluations were deliberately included to explore longer temporal dependencies, such as in audio, and to tackle more intricate spatio-temporal regression through self-supervision, as observed in optical flow tasks.

Answer 2: We have taken the reviewers' feedback into careful consideration and have supplemented our paper with a more extensive analysis of the computational and memory-intensive nature of BPTT in Appendix C. Although the analysis primarily discusses fully connected models, it can be extrapolated to encompass convolutional models as well. Additionally, we have included a visual representation in Figure 3 within Appendix C4 to illustrate a simple example that delineates how GPU memory scales linearly with the number of time steps, contrasting with the consistent of S-TLLR memory consumption. These additions aim to provide a comprehensive understanding of the computational and memory complexities associated with our proposed framework.

Answer 3: We sincerely appreciate the reviewers' valuable input regarding the additional papers. To ensure completeness and accuracy in our comparative analysis, we have incorporated the suggested papers in Table 3. It's essential to note that while these prior works focused on achieving high performance in image classification on static datasets, our emphasis lies in addressing temporal locality and memory-efficient training for Spiking Neural Networks (SNNs). The highlighted papers, while significant, do not encompass the pivotal aspects of our proposed approach, which emphasizes temporal dynamics and memory efficiency in SNNs

Answer 4: STDP, an unsupervised learning algorithm, operates by adjusting synaptic connections based on the relative timing of spiking activity but lacks a global supervisory signal to guide learning toward specific goals. So, we designed S-TLLR to incorporate a mechanism akin to STDP to identify neurons for update based on spiking activity. Then, we use a third factor—the learning signal—to attribute credit or assign blame to the neurons identified for update. This additional supervisory mechanism augments the synaptic plasticity process, enabling goal-oriented learning within the SNN framework.

Answer W1: We would like to address this question by stating that temporal locality means using only information available at the present. For example, in a sequence with multiple time steps, a temporal local operation at time step ($t$) must use only information available at that time step. By this definition, STDP expressed in equation (6) is a temporal local learning rule at any time step as it uses only information available at the present, and therefore its memory requirements are independent of the length of the sequence. In contrast, as we discuss in appendix C, BPTT does not present a temporal locality feature due to the backpropagation of errors trough time. This implies that the weight updates ($\Delta w$) at any time step depend on future information as shown in equation (23) and Figure 2a. Note that this happens due to the implicit recurrent connections (trough membrane potential accumulation) of hidden layers and not the loss function itself, so even if the loss for BPTT is computed as described in equation (11) and just for the last time step ($T_l=T$), propagation of errors is going to be similar to the one shown in Figure 2a with the small difference that the connections between the last layer and the loss function disappear, but still the weight updates at time step $t$ ($\Delta w [t]$) are going to depend on information at future time steps ($t+1, t+2, ..., T$). Therefore, BPTT is not a temporal local learning rule and requires the information of every single time step to compute the synaptic updates. In the case of S-TLLR, similar to STDP we can use trace variables to propagate the required information forward in time and use a learning signal obtained by propagation of errors trough the layers (not in time). Therefore, S-TLLR memory requirements are independent of the sequence length. An analysis on memory requirements is shown in Appendix C. Additionally, we would like to point out that equation (11) represent the instantaneous error signal obtained at the current time step ($t$), and the term $T_l$ only control at which point the targets are available and therefore the number of weight updates. So even if the learning signal is available for all time steps ($T_l=0$) the memory requirements of S-TLLR are the same.

Answer W2: In response to the ablation study suggestion, we conducted experiments, presented in Table 5 (Appendix E3), using three activation functions with varied $\alpha_{post}$ values due to resource constraints. The findings consistently demonstrate superior performance when using non-zero $\alpha_{post}$ compared to $\alpha_{post}=0$.

Answer W3: We have included a new appendix D elucidating the application of our method in recurrent models. Additionally, we updated the Figure 1 to ensure the discussion's generality.

Answer W4: We appreciate the feedback and have clarified that the primary text focuses on feed-forward networks, with discussions regarding recurrent models available in the supplementary material (Appendix D). This distinction aims to provide a comprehensive understanding while maintaining clarity within the main text.

Answer Q1: Yes, the blue terms in Figure 1 should be extend beyond $t-2$. We have updated Figure 1 to ensure the discussion's generality.

Answer Q2: In Appendix C, we presented the analysis to estimate the improvements and updated the reported improvements accordingly. Notably, we deliberately omitted specific details regarding the training duration of the models in GPU. This decision stems from our deliberate emphasis on explicitly illustrating the workings of S-TLLR within our implementation (code), prioritizing a comprehensive demonstration over runtime optimization. The variability in GPU training times is contingent upon specific code implementations, a facet that diverges from the primary focus of our discussion.

Answer 1: We have expanded our discussion on the number of multiply-accumulate (MAC) operations required by both S-TLLR and BPTT in Appendix C. While the estimation of dynamic energy consumption is typically proportional to the number of MAC operations, during training, the energy consumption associated with memory read and write operations might significantly contribute, particularly for memory-intensive methods like BPTT. Accurately modeling such energy consumption is a complex task beyond the current scope of our research. It's worth noting that S-TLLR was specifically designed as a three-factor learning rule, tailored for potential utilization on neuromorphic hardware like Loihi, which is an avenue we plan to explore in our future research.

Answer 2: Primarily, our focus lies on event-based tasks, as we believe these tasks inherently benefit from utilizing SNNs due to their temporal information, unlike static image classification tasks. While our exploration into SNNs hasn't delved extensively into deeper architectures, this limitation is largely due to the absence of neuromorphic datasets comparable in size to more mainstream vision tasks such as CIFAR100 or ImageNet. However, it's important to note our experiments on N-Caltech101, which contains a similar number of classes as CIFAR100, demonstrating that S-TLLR performs comparably to BPTT. Additionally, our experiments on optical flow using MVSEC, a challenging spatiotemporal regression task, show that S-TLLR can achieve performance similar to BPTT. These results instill confidence that S-TLLR can indeed be extended to deeper models and a broader spectrum of tasks.

Answer 1: Thank you for pointing out the similarities with OTTT [1], which allows for a more in-depth discussion. It's essential to note that while both algorithms share certain aspects, their fundamental motivations differ significantly. OTTT strives to approximate BPTT and is derived from BPTT equations by detaching all recurrent connections from the auto-differentiation graph. In contrast, S-TLLR is based on the theory of three-factor learning rules, utilizing the STDP mechanism to compute eligibility traces and a learning signal derived from backpropagation through layers or random feedback connections (DFA).

Furthermore, the selection of STDP in S-TLLR stems from the observation that the exponential decay of the causal term in STDP is akin to the computation of gradients over time with a decay factor equal to the leak factor of spiking neurons. However, while STDP considers both causal and non-causal terms for synaptic plasticity, focusing solely on causal information might result in a loss of valuable learning information. Hence, S-TLLR proposes a three-factor learning rule that models an STDP-like mechanism for neuron eligibility.

Another significant distinction lies in the target applications; OTTT, as stated by its authors, targets static image classification, while S-TLLR is specifically designed for temporal dependency tasks, deemed more suitable for SNNs. Notably, S-TLLR encompasses a pool of learning rules defined by the selection of hyperparameters (STDP parameters), where OTTT represents a specific case with hyperparameters $\alpha_{post} = 0$ and $\lambda_{pre} = \gamma$.

In our evaluations (Table 2, 3, 4, 5), using non-causal terms consistently yields better performance than using only causal terms (S-TLLR with $\alpha_{post}=0$ equivalent to OTTT). Even though we attempted to replicate OTTT's results for DVS CIFAR10 using S-TLLR ($\alpha_{post}=0$) in Table 3, our batch size discrepancy—using 48 compared to [1]'s 128 due to GPU memory limitations—may have influenced the outcome. Our comparison across various datasets supports the superiority of S-TLLR using non-causal terms over OTTT.

Answer 2: To clarify, we did not enforce the learning rule every fourth forward step but during the last $T-T_l$ timesteps of the sequence. For N-CALTECH101, DVS Gesture and DVS CIFAR10, it applies to the last 5 time steps, and for Optical Flow, it occurs at the final timestep. Following the reviewer's suggestion, we conducted simulations for DVS CIFAR20, presenting the learning signal during every timestep of the sequence, as indicated in Table 3. Consistently, S-TLLR with non-causal terms outperforms OTTT (S-TLLR with $\alpha_{post} = 0$) for the same batch size 48.

Answer 3: As previously discussed, OTTT can be viewed as a specific instance of S-TLLR when $\alpha_{post}=0$ and $\lambda_{pre}=\gamma$ (the leak factor in LIF models). Consequently, the ablation studies comparing OTTT and S-TLLR are encompassed across the work in Table 2, 3, 4 and 5. Our results consistently demonstrate that S-TLLR outperforms OTTT (S-TLLR with $\alpha_{post}=0$) under same conditions.