Forward Gradient Training of Spiking Neural Networks

Thank you for your valuable comments. We respond to your comments and questions as follows. The citations are the same as the paper unless specified.

1. About training costs on GPU.

First, our training cost comparison is with BP based on online training methods (Xiao et al., 2022) rather than BPTT. As described in the paper, our work is built upon online methods and the comparisons mainly focus on different spatial credit assignment methods given the online methods. For BPTT, the training memory cost is proportional to time steps, while online training methods have constant memory costs and is smaller than BPTT (e.g., for the considered 6 time steps, there can be $2-3\times$ reduction). Our method also has smaller memory costs than BPTT.

Second, we would like to emphasize that our method mainly focuses on developing neuromorphic-friendly algorithms rather than only considering efficiency on GPU. Our ultimate goal is to consider efficient neuromorphic hardware, and as it is currently immature, we develop methods with simulation on GPU. Our method has more neuromorphic properties and is more plausible than BP. And compared with random-feedback-based methods (DFA), our method significantly improves performance to a similar level as BP. We can realize these targets with comparable costs, which could be acceptable.

In Appendix E.1, we also discussed training costs conditions on potential neuromorphic computing hardware in the text, and our FGT (S) is better than BP (if possible) considering neuromorphic computing. On GPU, the training costs of FGT (or FGT(S)) is (slightly) larger than BP because GPU does not follow neuromorphic architectures and we do not optimize low-level codes of our method as BP in the established PyTorch libraries (the results are only brief comparisons). It can also be further improved, e.g., with parallelization of computation for forward gradients.

Meanwhile, our developed variant FGT (S) is more practical in real applications and also has about the same costs and speed as BP on GPUs, as shown in Table 6 in Appendix E, which can also scale to large-scale networks as shown in the following response to the third point.

2. About discussion of implementation on a chip or the associated storage and computational costs.

Currently, the development of neuromorphic hardware is immature and is also developing considering software-hardware co-design [1], so most algorithms are simulated on common computational devices while considering properties of neuromorphic computing, without experiments on real neuromorphic chips. The compatibility for implementation is measured by fitness to neuromorphic properties (or biological plausibility) because neuromorphic hardware will be designed following them for efficient event-driven and in-memory computation. FGT tackles BP’s problems of weight transport and separate phases for error signal propagation as it only requires unidirectional forward propagation across layers, conforming to properties of neuromorphic computing. We do not restrict the method to specific hardware, but follow the basic properties that will guide the development of hardware and algorithms. In Section 4.4, we also discussed some details on how the signal propagation of forward gradient can be made biologically plausible, proposing the variants of FGT (e.g., FGT (S), FGT (Q1, S)) which are also verified in experiments.

We have discussed storage and computational costs on potential neuromorphic computing hardware in Appendix E.1, and we further supplement a comparison table for clearer presentation in this revision. We quote the discussions and the table here: “for memory costs, … On the other hand, if we consider neuromorphic computing with unidirectional synapses, BP (if possible) should maintain all backward synapses between successive layers, while FGT, FGT (S), and DFA keep direct feedback connections from the top layer, which can be much smaller than BP since middle layers usually have much more neurons. As shown in Table 5, the memory costs of DFA and FGT (S) can be much smaller than BP”, “for operation numbers, BP requires a backward propagation across $N$ layers, DFA requires direct feedback to $N$ layers, FGT requires $N$ forward propagation of forward gradients for different layers (across $N, N−1, ..., 1$ layers) and direct feedback to $N$ layers, and FGT (S) requires a forward propagation across average $N/2$ layers and direct feedback to $N$ layers which is more practical than FGT. Direct feedback to $N$ layers may need fewer operations than BP across $N$ layers as middle layers usually have more neurons (for example, propagation from 400 to 400 neurons requires much more operations than from 10 to 400 neurons). So with a rough estimation, DFA needs much fewer operations than BP, and FGT (S) needs fewer operations than BP, as shown in Table 5”.

(continued below)

(continued from the preceding paragraph)

Considering neuromorphic computing, FGT (S) can be better and we quote our discussion and comparison table in Appendix E.1 here: “for memory costs, … On the other hand, if we consider neuromorphic computing with unidirectional synapses, BP (if possible) should maintain all backward synapses between successive layers, while FGT, FGT (S), and DFA keep direct feedback connections from the top layer, which can be much smaller than BP since middle layers usually have much more neurons. As shown in Table 5, the memory costs of DFA and FGT (S) can be much smaller than BP”, “for operation numbers, BP requires a backward propagation across $N$ layers, DFA requires direct feedback to $N$ layers, FGT requires $N$ forward propagation of forward gradients for different layers (across $N, N−1, ..., 1$ layers) and direct feedback to $N$ layers, and FGT (S) requires a forward propagation across average $N/2$ layers and direct feedback to $N$ layers which can be more practical than FGT. Direct feedback to $N$ layers may need fewer operations than BP across $N$ layers as middle layers usually have more neurons (for example, propagation from 400 to 400 neurons requires much more operations than from 10 to 400 neurons). So with a rough estimation, DFA needs much fewer operations than BP, and FGT (S) needs fewer operations than BP, as shown in Table 5”.

The above table (Table 5 in Appendix E.1) is an estimation of training costs on potential neuromorphic hardware. For illustration, we consider neural networks with $N$ hidden layers with $n$ neurons for each layer and $m$ neurons for the output layer, where $m \ll n$. The costs mainly focus on additional synaptic costs besides the normal forward procedure. Our method FGT (S) is expected to be better than BP considering costs, and achieving much better performance than DFA.

Additionally, we also would like to emphasize that our method mainly focuses on developing neuromorphic-friendly algorithms rather than only considering efficiency on GPU. We realize our targets, i.e., more neuromorphic properties and a similar level of performance as BP, with comparable costs, which could be acceptable.

3. About why FGT and LL are plausible.

FGT tackles BP’s problems of weight transport and separate phases for error signal propagation as FGT only requires unidirectional forward propagation across layers, conforming to properties of neuromorphic computing. This makes FGT more biologically plausible. LL is also more biologically plausible than BP because it is local learning without signal backpropagation across layers, avoiding the above problems of BP.

Considering what is not plausible, the original FGT requires propagating real-valued forward gradients for all layers, which may be hard to realize. So in Section 4.4, we have further discussed how the signal propagation of forward gradient can be made plausible with bursts and random sampling of layers, proposing the variants of FGT (e.g., FGT (S), FGT (Q1, S)) which are also verified in experiments.

4. About whether BP can be more hardware compatible than FGT.

While it is true that neuromorphic hardware is still being developed, BP is not a good choice because it is not compatible with the basic thought of neuromorphic computing and can have larger costs even if it is supported (e.g., frequent memory exchange, more synapses considering forward and backward between all successive layers, etc.), as discussed in the above response to the second point. Our FGT (and variants) is not specific to certain hardware, but follows the basic properties of neuromorphic computing that will guide the development of hardware, which may be more suitable. We believe that developing biologically plausible training methods is important for SNNs.

5. About connection to biologically-inspired learning rules.

In Section 4.3, we have discussed the connection to three-factor Hebbian learning. Our method is based on previous online training methods of SNNs, which is connected to the three-factor rule, while our method further answers how the global modulator can be determined – in our method, it is directly propagated from the top layer, corresponding to a more plausible neuro-modulation than previous methods that should be backpropagated across layers.

Thank you for your valuable comments. We respond to your comments and questions as follows. The citations are the same as the paper.

1. About novelty.

Actually, our key contribution is Section 4.2 where we propose a novel method for forward gradients, i.e., momentum feedback connections, while Section 4.1 is only our minor contribution that generalize the concept of forward gradient to the surrogate gradient condition of SNNs. The important problem of forward gradients in previous works (Silver et al., 2022; Baydin et al., 2022; Ren et al., 2023) is that the vanilla forward gradient is a projection to a random direction, so it suffers from large variance and performs poorly, which is also verified in our experiments. To tackle the problem, in Section 4.2, we propose the novel forward gradient with momentum feedback connections, which can largely reduce the variance and significantly improve the performance, also with theoretical analysis. With the proposed method, our FGT can achieve a similar performance as BP, which is not achieved by all previous works on forward gradients. This is one of our key contributions. Additionally, another novel contribution is presented in Section 4.4, where we also propose variants of our method with more biologically plausible considerations (e.g., FGT (S), FGT (Q1, S)) and we conduct experiments to verify their effectiveness, which has not been considered by previous works. Our method is novel and different from existing forward gradient methods.

2. About results considering hardware compatibility and training costs.

Currently, the development of neuromorphic hardware is immature, so most algorithms are simulated on common computational devices while considering properties of neuromorphic computing, without experiments on real neuromorphic chips. The compatibility is measured by fitness to neuromorphic properties (or biological plausibility) because neuromorphic hardware is designed following these properties for efficient event-driven and in-memory computation. Note that an important feature of neuromorphic computing is unidirectional synaptic connections for event-driven and in-memory computation to avoid frequent memory transport which accounts for the majority of energy costs in real hardware (the computation architecture is expected to be different from the commonly used hardware with von Neumann architecture such as CPU or GPU). This means that each synapse between two neurons only allows unidirectional communications, and its weight value is stored locally. This is the reason why BP can hardly be suitable for (future) neuromorphic hardware, because BP will require two different forward and backward connections between successive layers with exactly the same weight, which will need frequent memory exchange (and also two stages, etc.) that is not compatible to the basic thought of neuromorphic computing. Our proposed FGT (and variants), on the other hand, avoids BP’s problems and is thus more biologically plausible and compatible with neuromorphic hardware.

As for the discussion on training costs in Appendix E, Table 6 is only a rough comparison on GPU and we have also discussed conditions on potential neuromorphic computing hardware in the text, where our FGT (S) is better than BP (if possible) considering neuromorphic computing. We also supplement a comparison table (Table 5 in the revised version, also shown below) for clearer presentation of this. On GPU, the training costs of FGT (or FGT(S)) is (slightly) larger than BP because GPU does not follow neuromorphic architectures and we do not optimize low-level codes of our method as BP in the established PyTorch libraries (the results are only brief comparisons). It can also be further improved, e.g., with parallelization of computation for forward gradients.

Meanwhile, our developed variant FGT (S) is more practical in real applications and also has about the same training costs and time as BP on GPUs, as shown in Table 6 in Appendix E.1.

(continued below)

Thank you for your valuable comments. We respond to your comments and questions as follows. The citations are the same as the paper unless specified.

1. About discussion and comparison to works with SOTA performance.

Our work is a different line from these works, and they can be orthogonal and combined with each other. Specifically, recent works with SOTA performance focus on training high-performance SNNs on common hardware (e.g., GPU) with training methods such as BPTT and other improvements of models, and the trained models are expected to be directly deployed on neuromorphic hardware. Their hardware efficiency is about trained models. Our work, on the other hand, aims at developing algorithms that adhere to neuromorphic properties and are expected to pave paths for on-chip training of SNNs. Our hardware efficiency is about the training algorithm.

These two lines of work are compatible with each other. First, most recent SOTA works focus on the improvement of models and are still based on BP for training. Our work considers a different error propagation method and is orthogonal to them, which can be possibly combined in the future. With biologically plausible training methods that pave paths for potential on-chip training, we may also expect energy-efficient training directly on hardware, because GPUs are not designed for event-driven and in-memory computation, and training SNNs on GPUs is costing.

Second, considering real applications, we can also first pretrain a high-performance SNN on GPU as these works, and after deploying it on hardware, we may expect on-chip training of the models to deal with problems such as hardware mismatch or tuning the model for new data or tasks so that the model can adaptively evolve for real applications.

To demonstrate the effectiveness for such scenario, we supplement an experiment on finetuning a pretrained ResNet-34 on ImageNet under potential hardware noise. This task is on the ground that there can be hardware mismatch for deploying trained SNN models (Yang et al., 2022; [1]), and we may expect finetuning models directly on hardware to better deal with the problem. We simulate a kind of potential hardware noise by considering the stochastic neuron setting as described in Section 3.1, which may correspond to some hardware noise such as thermal noise. We consider finetuning a pretrained ResNet-34 model released by Xiao et al. (2022), which is trained under the deterministic setting (i.e., no noise) with the original test accuracy as 65.15%, under the noise setting with different levels of noises. All methods are trained for only 1 epoch and the results are below (where the noise parameter corresponds to the hyperparameter for the sigmoid function of the logistic noise, the larger the parameter the larger the noise scale).

As shown in the results, our proposed FGT (S) can effectively finetune such large-scale model, while previous biologically plausible methods with random feedback (DFA) totally fail. BP is not biologically plausible and its results are only for reference. FGT (S) has lower performance than BP under the condition of 1 epoch because it may have a slower convergence rate than BP (as studied in Section 5). Our proof-of-concept results mainly verify the effectiveness of our method for such scenarios and large-scale models. Future work can consider more SOTA models.

We do not overstate our contribution as we do not aim at outperforming models with SOTA performance, but propose a new biologically more plausible training algorithm and show that it can achieve similarly promising results as BP which is adopted by training those models. Our experiments also mainly make fair comparisons with BP under the same settings and show the effectiveness of our method with better neuromorphic properties, without overstating superiority over SOTA models.

We have added the above discussion and experiments, and cited your provided valuable references in the revised paper.

2. About the method’s superiority.

Our method is superior to BP mainly because it tackles BP’s problems of weight transport and separate phases for error signal propagation as it only requires unidirectional forward propagation across layers, conforming to properties of neuromorphic computing. At the same time, our method can achieve comparable performance as BP, largely outperforming previous biologically plausible methods with random feedback. In short, the advantage lies in promising performance with better properties, paving paths to biologically plausible on-chip training of SNNs.