DAS-GNN: Degree-Aware Spiking Graph Neural Networks for Graph Classification

Q3: As mentioned in the paper, the real-world graphs tend to exhibit an extremely skewed distribution of degrees. The issue also remains in many large-scale graphs. Providing extra experiment results on node classification tasks would enhance the persuasiveness of the paper.

$\to$ We provide additional experiment results on the large-scale graphs for both graph classification and node classification tasks. We used REDDIT-BINARY and COLLAB datasets, which are large enough to exhibit a skewed distribution of degrees in the graph classification tasks. We show that DAS-GNN still outperforms when applied to large-scale graph classification tasks.

In addition, we experimented with two node classification datasets: ogbn-arxiv, and ogbn-mag. DAS-GNN still performs the best, but the gap is smaller compared to the graph classification tasks. We believe this is because the node classification depends more on per-node properties rather than the graph structures, benefitting less from the proposed techniques.

Q4: DAS-GNN energy consumption is not only related to the firing but also to the grouping strategies. It would be clearer if the authors could provide the energy consumption (in the formula) and further discuss the impact of the aforementioned settings on the energy consumption.

$\to$ Yes, it is true that we need a small additional cost for updating the threshold per group. The overhead for computing the adaptive threshold operation would be $2 \cdot E_{AC} \times$ num_groups $\times$ hidden_dimension to account for the two additional multiplications related to $\gamma$per group in Eq. 10. We modified the energy consumption table to consider the operations needed for updating the thresholds, which demonstrates the small overhead. Please also find it in section 5.6 and Appendix J of the revision.

Q5: GNN also shows highly diversified spike distributions on MUTAG and ENZYMES datasets. But MUTAG-PGNN-GCN and ENZYMES-PGNN-GAT remain obvious performance gaps.

$\to$ We believe this is related to a trend we found where the spike rate diversity of the ultimate layer has an especially high correlation with the performance in general. In both the PGNN-MUTAG-GCN and PGNN-ENZYME-GAT, low diversity is observed ultimate layer (layer 3 for GCN, layer 2 for GAT). Although an exact reason is up for further investigation, we believe this is a reasonable behavior because it is directly used in the final output. We have discussed this in Section 5.5 and Figures 27-28 of the revision.

We appreciate acknowledging the strength in our work and providing detailed feedback.

W1: Some descriptions of neurons and nodes are unclear. In Section 3, neurons are related to the feature dimension. But, in Section 4.2 the paper claims that neurons can be grouped by their degrees

$\to$ The neurons belonging to vertices of the same degree and the same feature position are grouped together. This yields groups shaped as vertically long bars as newly depicted with green boxes in Fig. 1(b) of the revised paper. We apologize for the confusion and moved the description regarding feature splitting to appear earlier in the revision.

W2: The definition of the “degree group” is vague and the paper lacks more explanations about the groups N_g or grouping strategies.

$\to$ A degree group comprises multiple degrees that can be found within an input graph. It is a concept we introduced in Appendix E to represent a case where we would want less number of groups than the number of unique degrees. We provided the definition in the revision. In addition, $N_g$ denotes the set of neurons within group $g$. We also clarified the definition in the revision.

W3: Confusion of categorizing baselines and lack of details on the settings of these baselines.

$\to$ All baselines are spiking neuron variants that are compared against our method at the neuron-level. To avoid confusion, we replaced their names in Table 1 with the type of neurons (e.g., Vanilla LIF neuron for SpikingGNN and adaptive techniques added neuron for SpikeNet). We apologize for the concern caused by the ambiguous terminology.

We added experimental architecture details in the experiment settings on how we configured the baselines Appendix B and Section 5.1, such as the number of layers and hidden dimension size.

Q1: For those graphs following the power-law distribution, simply dividing nodes by their degrees may result in an uneven number of nodes between different groups. How does the grouping strategy proposed in the paper alleviate this issue?

$\to$ Yes, DAS-GNN could result in an uneven number of nodes between groups. However, we believe this would not be very problematic. Although the size of each group could be large, the neurons within the group will exhibit similar firing rates. Considering that the purpose of grouping is to share a threshold together among neurons with similar firing rates, we found that this empirically does not cause a problem. Please also see Table 7 for results with larger graphs.

Q2: In the experiment, the new and relevant spiking graph neural networks and spiking neuron variants can be introduced separately as baselines to demonstrate the model’s effectiveness. Additionally, the authors should provide architectural details of baselines built from spiking neuron variants.

$\to$ We added [4] to our main table (Table 1) as ‘GLIF’. The tendency of the result did not change after we added a new baseline (GLIF).

Additionally, we conducted experiments with the three proposed techniques from [2-3], with and without techniques from DAS-GNN as shown in the table below. It shows that DAS-GNN is orthogonal to the techniques, and can be used to further improve the performance in all the tested cases. The result can be also found in Appendix K.

In addition, we added architectural details of baselines on how we configured the baselines in Appendix B.

We thank the reviewer for acknowledging the novelty of the work and providing constructive feedback. We have addressed the comments below.

W1: The proposed degree-aware group-adaptive neurons require the thresholds to depend on other neurons. Is it plausible for potential neuromorphic hardware? Or will it introduce much more computation for communications between neurons?.

$\to$ Yes. Our method would indeed be plausible for potential neuromorphic hardware [1-3] without introducing significant computational overhead for communication between neurons. Neuromorphic processors are typically designed with neuron cores as fundamental units, where each core handles neuron states (e.g., membrane potential) stored in local memory (often an SRAM), updates them based on synaptic inputs, and generates spikes upon reaching a threshold. To implement DAS-GNN, each neuron group can be placed together in a core as a whole. Then, the sum of firing rates (Eq. 9) can reside in the local memory and be updated by counting spikes from each neuron of the same group. Since the threshold adjustment happens locally at the core level, it doesn’t need extensive communication overhead between neurons. Please find this discussion in section 5.6 of the revision.

[1] Merolla, Paul A., et al. "A million spiking-neuron integrated circuit with a scalable communication network and interface." Science 345.6197 (2014): 668-673.

[2] Akopyan, Filipp, et al. "Truenorth: Design and tool flow of a 65 mw 1 million neuron programmable neurosynaptic chip." IEEE transactions on computer-aided design of integrated circuits and systems 34.10 (2015): 1537-1557.

[3] Davies, Mike, et al. "Loihi: A neuromorphic manycore processor with on-chip learning." Ieee Micro 38.1 (2018): 82-99.

W2: It is not clear if the energy consumption analysis takes the costs of this adaptive threshold operation (and potential communications between neurons) into account.

$\to$ We updated the energy consumption analysis to include the adaptive threshold operation. As mentioned in W1, DAS-GNN does not incur any additional communication. The overhead for computing the adaptive threshold operation would be $2 \cdot E_{AC} \times$ num_groups $\times$ hidden_dimension to account for the two additional multiplications related to $\gamma$per group in Eq.10. However, as shown in the table, this only adds a minimal amount of overhead to the energy consumption. This was also added to section 5.6 of the revised version.

Q1. For the proposed method, is only the spike rate considered?

$\to$ We agree that considering temporal information could further enhance the performance of DAS-GNN. For example, we could apply a similar idea by assigning synaptic delays based on the degree of each node, or encode community information as a temporal property. We added the discussion in Appendix K.

Q2. Sensitivity for other parameters

$\to$ We provide additional sensitivity studies regarding $\gamma$ from Eq. 10, and the size of GNN hidden dimensions. We find that DAG is generally insensitive to $\gamma$, with minimal degradation within the given range. We use 0.2 as the default value, which is generally the best setting across different model architectures. For the study on GNN hidden dimension size, we find that our method is insensitive to varying sizes of GNN, consistently outperforming PLIF neurons across different hidden dimension sizes. Please also find the results in Table 3. and Appendix H, Appendix I of the revision.

Table: Sensitivity study on $\gamma$

Table: Sensitivity on layer hidden dimension DAS-GNN

Table: Sensitivity on layer hidden dimension PLIF

We thank the reviewer for acknowledging our contributions and giving positive feedback. We try our best to faithfully address the comments below.

W1: Lack of the theoretical justification of the DAG method.

$\to$ Although a complete analysis would be our future work, we provide a rough theoretical justification for grouping neurons based on their degree. By reformulating the message passing part of Eq.7 in the perspective of a single neuron, the membrane potential of a neuron $i$ is computed as $U_i=\sum_{j\in active(i)}W_{i,j}$, where $active(i)$ denotes firing neurons among neighbors of $i$. When we follow a common assumption where $W_{i,j} \sim N(0,\sigma)$, it leads to $U_i \sim N(0, \sigma \cdot |active(i)|)$. Since $U_i$ determines the firing rate and $|active(i)|$ is proportional to the degree of $i$, grouping neurons by its vertex degree has an effect of grouping neurons with similar membrane potential variance. This aligns with our findings from Section 3, and we added this analysis in Section 4.2 of the revised paper.

W2: Simplifying mathematical notations and providing detailed explanations for spike mechanism and model architecture.

$\to$ We simplified some mathematical notations in Eqs.8-10 and clarified explanations. In addition, we added model architecture details for experimental settings. A major change is that we changed the symbol for membrane potential from $V(t)$ to $U(t)$, which could have caused some confusion with the threshold voltage $V_{th}$. In addition, we added our proposed grouping method in Figure 1(b) with green boxes for more intuitive understanding.

Q1. Scalability to a very large graph with skewed degree distribution

$\to$ We show additional experiment results on two large-scale graph datasets used for graph classification: REDDIT-BINARY and COLLAB. We could find that our DAS-GNN still outperforms other baselines in large graphs with skewed degree distribution. Please also find these results in Appendix D of the revision. For another demonstration on performance on large graphs, we extend our method to node classification datasets (ogbn-arxiv, ogbn-mag), where the results further show that DAS-GNN maintains its competitive performance.