Delay-DSGN: A Dynamic Spiking Graph Neural Network with Delay Mechanisms for Evolving Graph

Thank you for your meticulous review of our work and the valuable feedback provided.

Response to Weaknesses and Question 1:

In our model, delay parameters are randomly initialized following a normal distribution within a specified range. For example, when the maximum delay is set to 5, the initial delays are sampled from a normal distribution over the interval $[0, 5]$. Training results on the DBLP show that the learned delay distribution is slightly shifted to the right, indicating that academic collaborations and influences often require a longer time to become evident. For the Tmall, the learned delay distribution is slightly skewed to the left, reflecting the typically rapid user responses and interactions in e-commerce environments. The Patent exhibits larger delays, with a more uniform distribution toward the right, suggesting that the impact of patents generally takes a longer time to diffuse and manifest. Despite sharing the same initialization distribution, the model consistently learns dataset-specific delay values across multiple training runs, demonstrating its ability to adapt to inherent temporal characteristics.

Response to Theoretical Claims:

Thank you for your valuable feedback. We have added an extra analysis in the experimental section and visualized the training loss curves. Unfortunately, we are unable to display the figures here, so we provide the following explanation. When $σ$ is outside the derived range, the model's loss during training on the DBLP remains largely unchanged. In contrast, when $σ$ is within the derived range, the training loss consistently decreases, indicating that the model is able to learn and optimize effectively.

Response to Essential References Not Discussed and Question 2:

We have added a new paragraph in the related work section as follows: "Unlike the memory module in TGN, our delay mechanism explicitly models multi-step dependencies through differentiable temporal kernels, thereby mitigating the gradient decay issues inherent in RNN architectures. Compared to the Hawkes process used in TREND, the biologically inspired spike-delay mechanism in Delay-DSGN is better suited for handling discretized timestep-based graph updates. " We have compared our approach with attention-based methods such as TGAT. We will endeavor to include experimental comparisons with TGN to further demonstrate the effectiveness and advantages of our proposed method.

Response to Other Comments Or Suggestions:

Thank you for your valuable feedback and careful corrections on our paper. Based on your suggestions, we have carefully revised and improved the issues mentioned in the text as follows:

• In line 15 on page 2, “effectively mitigating historical information forgetting” has been revised to “effectively mitigating the forgetting of historical information” for improved clarity and grammatical accuracy.
• At the location of Equation (5) on page 4, we have added a reference to Equation (4) to better clarify the relationships between the different parts of the formulas.
• Regarding the comparison with eight methods but only seven results being presented in the experiments, we confirmed that this was due to the omission of one method in the table. We have now updated the table to include all the mentioned methods. Due to character limitations, the results of the omitted comparison methods on the three datasets are as follows:

Thank you for your meticulous review of our work and the valuable feedback provided.

Response to Experimental Designs or Analyses 1:

In our experiments, we compared eight methods across three datasets using two metrics. To assess the statistical significance of improvements by our proposed method over the baselines, we conducted Friedman tests. The results are:

With all P-values are less than the significance level $\alpha = 0.05$, indicating that our method's average ranking is significantly different from the baselines for both metrics.

Response to Essential References Not Discussed:

We have made additions to the "Related Work" section: "GC-LSTM integrates LSTM into GCN to model the temporal evolution of node features. It captures temporal dependencies through hidden state transitions. In contrast, Delay-DSGN introduces a delay convolution kernel to explicitly parameterize propagation delays. DyRep utilizes temporal point processes to model the triggering intensity and time intervals of events. Its core lies in describing the probabilistic patterns of event occurrences rather than capturing the dynamic delay effects of information propagation."

Response to Weaknesses:

Please refer to the first response to Reviewer WWHS.

Response to Concerns:

• Regarding zero-padding, convolution processes: For example, at the current timestep, two neurons from the previous layer aggregate information to one neuron in the current layer. The inputs are spike trains from these two neurons, each containing activity information across four timesteps (the current timestep and three historical timesteps). Neurons exhibit different delays. Before convolution, the input sequences undergo left-side zero-padding (with padding length equal to $K_s−1$) to accommodate the convolution kernel size. Convolution kernel construction: The hyperparameter $K_s$ represents the size of the delay convolution kernel, and $K_s−1$ is the maximum delay time. The values in the delay convolution kernel follow a Gaussian distribution, with the center position located at $K_s−d−1$. Subsequently, the padded sequences are convolved with the delay-specific kernels constructed between each pair of neurons. This operation fuses weighted information across timesteps, generating a new output spike sequence that aggregates information from two neurons to one.
• Regarding the computational cost of larger delay windows: This does not increase the computational cost. A larger delay window means a larger delay convolution kernel, which results in more left zero-padding, but the number of convolution computations remains unchanged.
• Rationale for Choosing the Gaussian Kernel:
  • Learnable Temporal Shift: The parameter $μ$ controls the central position of the delay, enabling the model to explicitly learn the delay magnitude. The smoothness of the Gaussian kernel ensures gradient stability during training.
  • Adjustable Receptive Field: The variance parameter $σ$ modulates the receptive field of the delay kernel, allowing it to capture multi-step historical information.
• Network Architecture: The dynamic graph is divided into fixed-interval snapshots. For each snapshot, second-order neighborhood sampling captures multi-hop dependencies, and first-order aggregation integrates local features. Features are encoded into spike representations using SNNs, converting continuous data into spike trains. Second-order aggregation incorporates broader context. In the Delay-SNN layer, the model dynamically adjusts the Gaussian delay kernel's center to capture varying temporal delays, generating node delay representations. Multi-timescale fusion integrates representations across timesteps, producing final embeddings used for downstream tasks.
• SNN-GNN Interaction and Benefits of SNNs for Dynamic Graphs: After traditional GNN-based neighborhood aggregation, the threshold-triggering mechanism of SNNs replaces conventional nonlinear activation functions. Advantages of SNNs for Dynamic Graphs: SNNs inherently accumulate membrane potentials across timesteps, aligning naturally with the discrete timestep nature of dynamic graphs. At each timestep, node information is stored in the membrane potential of SNN neurons. This potential is inherited to the next timestep, preserving historical states and enabling the modeling of graph evolution.

Response to Weaknesses Theoretical Claims:

Thank you for your valuable feedback. Your insights have provided us with an important perspective. To further explore this issue, we plan to conduct more detailed experiments focusing on: Evaluating our method's performance on dynamic graphs with diverse temporal patterns and topological structures. Testing our theoretical findings with additional real-world datasets to ensure their validity and applicability in practical scenarios.

Thank you for your meticulous review of our work and the valuable feedback provided.

Response to Weaknesses 1 and Question 1:

When a new edge is added to a graph, it may indeed represent an immediate interaction demand. However, in reality, there is usually a time lag between "establishing a connection" and "producing an effect." For instance, in social networks, after user A posts content, user B may take several days to respond; in traffic networks, congestion propagation requires time. These phenomena indicate that delays in information propagation are an inherent property of dynamic graph evolution. Therefore, our motivation is to explicitly model these delays to more accurately capture the dynamics and temporal dependencies. Delay-DSGN adaptively learns the delay parameters $\mu_{ij}$ within the delay kernel, dynamically determining appropriate delay value. This ensures that the speed of information propagation adapts to different application scenarios while preventing excessive delays.

Response to Weaknesses 2:

Neither our model nor the fixed random delay model includes a decay value for random initialization. The key difference lies in whether the delay parameters are learnable or remain fixed after initialization.

Response to Question 2:

When the number of synaptic connections in a neural network layer far exceeds the number of possible discrete delay positions, randomly initialized delays can cover the entire effective time range. For example, with 10 possible delay positions and 1000 synaptic connections, all delay positions will be covered. Thus, the necessity of moving delay positions away from their initial state diminishes. In this way, a fixed random delay model can achieve good performance by optimizing connection weights to capture temporal characteristics.

Response to Weaknesses 3:

• The core innovation of Delay-DSGN lies in its first-ever deep integration of biologically inspired delay mechanisms with dynamic graph modeling, along with rigorous theoretical guarantees for model stability. Specifically, Delay-DSGN explicitly models multi-step dependencies using learnable Gaussian delay kernels, capturing the relationship between topological connection strengths and signal propagation delays in dynamic graphs. Additionally, it provides strict upper bounds for gradient propagation in the spatiotemporal domain driven by delay kernels, offering theoretical guidance for model parameter selection. Compared to traditional SNNs and GNNs, Delay-DSGN not only enhances biological fidelity and temporal processing for SNNs but also introduces a groundbreaking delay perspective to dynamic graph modeling methods.
• Although current research mainly focuses on SNNs, the delay mechanism can be regarded as a type of temporal convolution method. This approach shows potential for extension to other types of dynamic models when dealing with time-series data. Future work will further explore the application scope of this mechanism.

Response to Question 3:

The delay convolution kernel primarily addresses the weighted aggregation of historical information within a single time step. However, topological changes in dynamic graphs often span multiple time steps. By integrating node representations from different time steps into a unified feature, it becomes possible to better capture long-term dependencies and complex topological evolutions. Secondly, this approach compensates for the sparsity of spike signals, mitigating information loss caused by the sparse nature of spike signals.

Response to Question 4:

Since Delay-DSGN introduces additional delay time parameters, to balance the total number of parameters, we increased the number of hidden neurons in the no-delay model. Additionally, we conducted experiments on a standard SNN model without adding extra neurons. The comparative experimental results are as follows:

Response to Other Comments or Suggestions:

• Equation (4) describes the node features processed with delay, which are subsequently used as input in Equation (5).

• We understand your questions about Equations (9) and (15) mainly concern the notation $w$，and we apologize for not providing sufficient explanations in the original manuscript. Here are the specific modifications:

  • For Equation (9), $w_t$ represents the weight of the features at time step $t$, which is obtained through an element-wise multiplication of $Z_v^t$ and $w_t$, followed by a row-wise summation.

  • For Equation (15), $w_{ij}$ represents the weight between neuron $i$ and neuron $j$.