Interactions Exhibit Clustering Rhythm: A Prevalent Observation for Advancing Temporal Link Prediction

Many thanks to Reviewer TU2Y for the thorough comments. We have revised our paper accordingly.

W1-1: Explanation of "TGNs".

Quick Answer: We would like to clarify that "TGNs" is widely used in the community, and we follow such terminology for readability.

• Consistency with the research community. "TGNs" is a commonly used terminology to refer to a family of models designed for learning node representations in temporal graphs [1, 2, 3].

• Differences between "Graph" and "Network." "Graph" generally refers to the structured data with nodes and edges, while "Network" typically describes the neural network in model structures such as KAN (Kolmogorov-Arnold Networks) [4].

[1] Towards Adaptive Neighborhood for Advancing Temporal Interaction Graph Modeling, KDD, 2024.
[2] Provably Expressive Temporal Graph Networks, NeurIPS, 2022.
[3] Adaptive Data Augmentation on Temporal Graphs, NeurIPS, 2021.
[4] Kan: Kolmogorov-arnold Networks, Arxiv, 2024.

W1-2: Illustration of the motivation for our work.

Revision: As revised in Lines 49-68, we have clarified the motivation for our work.

We provide the overall explanation of our motivation here.

• Limitations of existing works. As noted in Lines 49-68, existing works often focus on designing different model architectures to parameterize the interaction patterns in temporal graphs. Although powerful, these frameworks rely heavily on intricate computational processes to approximate their parameterized interaction patterns. Such a procedure tends to overlook the potentially heuristic, realistic patterns inherent in the raw data, leading models to depend increasingly on more complex architectures to fit fragmented details for performance improvement.
• Research gap between our work and existing works. Unlike the mainstream idea of designing complex model architectures, we introduce statistical empirical analyses and present the first attempt to investigate the heuristic temporal correlation among interactions. Building on these observations, we propose the first model that explicitly captures burstiness information for temporal link prediction.

W1-3: Illustrative examples for temporal clustering.

Revision: As revised in Line 66 of Figure 1, we have included an illustrative example that vividly demonstrates temporal clustering patterns.

Thanks for your suggestion! In Figure 1, we provide a toy temporal graph and plot the interaction rhythms for three example nodes. This example demonstrates how the behavior of a specific node in temporal graphs exhibits burstiness, providing a clearer and more intuitive understanding of temporal clustering.

W1-4: Explanation of the measurement and conclusion in empirical analyses.

Quick Answer: We would like to clarify that we have already introduced the measurement for temporal clustering and performed a reliable analysis to derive our conclusions in empirical analyses.

• Measurement for temporal clustering. As stated in Line 187, to quantify temporal clustering, we compute the average inter-event times for interaction nodes (notably, we have replaced the term "silence duration" with "inter-event times" to improve readability). Besides, in Line 190, we explain the motivation behind this method: "Interaction nodes exhibiting strong temporal clustering tend to experience shorter inter-event times."

• Analysis for conclusions. As mentioned in Lines 200-208 and 236-251, we have provided a thorough analysis to draw the conclusions. For example, as discussed in Line 201, most truly existing interaction nodes (the red distribution in Figure 3(a)) exhibit heavy-tailed distribution with shorter inter-event times. This indicates that nodes in temporal graphs reveal burstiness in their behaviors with shorter periods of inactivity, which strongly supports the presence of temporal clustering.

W1-5: Relevance between empirical analyses and link prediction.

Quick Answer: Kindly note that the comparison between truly existing and randomly sampled interaction nodes directly aligns with the task of distinguishing positive and negative links in link prediction.

• Relevance to link prediction. As outlined in Lines 154-159, we perform empirical analyses by comparing the differences between truly existing and randomly constructed interactions. This setup directly corresponds to the link prediction task, which aims to distinguish positive links (those that truly exist) and negative links (those that do not).
• Insights for link prediction. As revised in Lines 243-245, we summarize the key factors of temporal clustering for temporal link prediction and explain them as (i) if a given node has recent interactions, then it is more likely to interact at the current timestamp; and (ii) if the node has only distant past interactions or even no historical interactions, it is less likely to interact now.

W2-1: Explanation of the motivation behind neighbor selection strategy.

Quick Answer: We would like to clarify that we do not claim that existing works fail to preserve interaction rhythm patterns. Rather, their limitation lies in "incorporating spurious or noisy information", which prevents them from effectively preserving the latest and most relevant interaction patterns.

In Lines 264-268, we have acknowledged that the existing neighbor selection strategy can help preserve interaction rhythm patterns. The limitation of these techniques is that "they may risk incorporating spurious or noisy information either from high-order connections or long-outdated histories". To this end, we sample the most recent historical links, preserving the latest, most relevant interaction patterns for accurate future link prediction.

W2-2: Explanation of the sample size $m$.

We kindly draw your attention to Line 280, for nodes with fewer than $m$ historical links, we select all available links and pad the corresponding matrix to $m$ dimensions. This technique is also adopted in existing studies [1, 2].

[1] Towards Adaptive Neighborhood for Advancing Temporal Interaction Graph Modeling, KDD, 2024.
[2] Towards Better Dynamic Graph Learning: New Architecture and Unified Library, NeurIPS, 2023.

W2-3: Explanation of $\mathbf{C}_{\text{rhythm}}^t$.

Kindly note that, as mentioned in Line 305, the rhythm vector $\mathbf{C}_{\text{rhythm}}^t$ is shared communally across all nodes and will be updated globally and chronologically. Therefore, to avoid confusion, we do not use the superscript $u$.

W3: Explanation of theoretical analysis.

Quick Answer: We kindly believe that the theoretical analysis provides a strong foundation for supporting the proposed silence decay mechanism.

In the theoretical analysis presented in Section A, we define the Hawkes process and temporal clustering effects, which form the basis for deriving Eq. 8. Our silence decay mechanism, introduced in Eq. 9, is closely aligned with the framework outlined in Eq. 8.

As shown in Eq. 9, when generating the representation of node $u$ at timestamp $t$, we retrieve $\mathbf{C}_ {\text{rhythm}}^t$ and use it as the underlying rate of interaction occurrences ($\mu_u^t$ in Eq. 8). The decay function $\text{g}(\cdot)$ acts as the kernel function to capture the time decay effects ($\rho(\cdot)$ in Eq. 8), and the $\mathbf{C}_ {\text{temp}}^t$ represents the excitation amount for future behavior of node $u$ ($\gamma_u^{t'}$ in Eq. 8). For further details, please refer to Lines 756-801.

W4: Clarification of Figure 3 (Figure 2 in the original paper).

Definition of X-axis. We would like to clarify that as noted in Lines 160-188, we have provided the formulated definition of truly existing and randomly sampled interaction nodes in our original paper. Specifically, truly existing interaction nodes refer to node pairs that exist in temporal graphs, while randomly sampled interaction nodes are obtained by substituting the original interaction nodes with randomly sampled nodes. For further details, please refer to Lines 160–188.

Meaning of "macro-level". Kindly note that as mentioned in Lines 250-257, we have already explained the meaning of "macro-level" in our original paper. To be specific, we claim that macro-level analyses capture node interaction rhythms across the entire timeline, rather than focusing on micro-level time steps. This is why we also conduct micro-level analyses for more granular insights.

Units of Y-axis. Regarding the units of inter-event times, some datasets, such as Wikipedia, normalize timestamps when they are made publicly available. As a result, the timestamps in these datasets do not have explicit units. Furthermore, as shown in Table 6, the time granularities vary across different datasets. Therefore, following the analyses in [1], we focus solely on the numerical values of the timestamps without considering units.

Thank you once again for your invaluable comments. Your efforts have been instrumental in helping us improve our work significantly.

[1] Towards Better Evaluation for Dynamic Link Prediction, NeurIPS, 2022.

Many thanks to Reviewer AAW9 for providing an insightful review.

W1-1: Explanation of temporal clustering.

Quick Answer: Kindly note that temporal clustering studies the burstiness of interactions for specific nodes rather than node popularity.

Difference between burstiness and popularity (node degree). Bursty interactions within short periods emphasize relative interaction frequencies compared to other periods and are not directly correlated with node degree. For example, as mentioned in Line 77, a user's purchasing behavior might occur around weekends while exhibiting few activities during weekdays, even if the total number of purchases of this user is low.

Insights of temporal clustering for link prediction. As revised in Lines 242-245, temporal clustering encodes the following insights for link prediction:

• (i) If a given node has recent interactions, then it is more likely to interact at the current timestamp;
• (ii) If the node has only distant past interactions or even no historical interactions, it is less likely to interact now.

W1-2: Evaluation under different negative sampling strategies.

Quick Answer: TG-Mixer performs best across four negative sampling strategies, reinforcing its effectiveness.

Revision: We have incorporated these discussions in Lines 972-1007.

Empirical Analysis. In Line 908, we evaluate model performance using random sampling strategy. We note that recent work [1] has introduced advanced evaluation methods for temporal link prediction. To fully assess the models, we adopt a more challenging evaluation with four negative sampling strategies and rank-based metrics. Descriptions of these strategies are put in the "Implementation Details".

Experimental Evaluation. Due to char limitations, we provide key findings and results here. Other analyses can be found in Lines 972-1007.

• TG-Mixer demonstrates strong performance across four negative sampling strategies, further proving its robustness.
• Inductive negative sampling often leads to the most performance drop. This is likely because this strategy samples unseen nodes to construct negative links, making it more challenging for models to distinguish temporal links.

Table 1: A subset of transductive MRR results on Reddit.

Implementation Details. In addition to random negative sampling, we follow [1] and use historical negative sampling (sampling negative links that have been observed) and inductive negative sampling (sampling negative links that are unseen during training). To mitigate potential biases from popular nodes, we conduct degree-aware negative sampling by sampling negative nodes with the probability of $d/N$, where $d$ is the node's current degree and $N$ is # historical interactions.

[1] Towards Better Evaluation for Dynamic Link Prediction, NeurIPS, 2022.

W2: Supplementary material.

Kindly note that we have updated the complete codes for TG-Mixer. Please check it.

Q1: Explanation of negative sampling.

As revised in Lines 908-911, we follow the same configurations as DyGLib, where we fix the source nodes and sample an equal number of destination nodes to construct negative links. Thus, our negative and positive edges share the same source nodes.

Q2: Evaluation on more datasets in TGB.

Quick Answer: TG-Mixer exhibits superior performance across the additional datasets in TGB.

Empirical Analysis. In the original paper, we conduct experiments on the two selected datasets because many baselines may not be implemented on other datasets due to computational issues ('NA' in the following Table). To further validate the model performance, as revised in Line 956, we add three datasets: Review, Comment, and Flight.

Experimental Evaluation. We provide a subset of experimental results here. For the complete results, please refer to Table 7 of Line 956.

• TG-Mixer consistently demonstrates superior performance on the additional datasets, highlighting its robustness and strong capabilities.
• The most significant performance improvements are observed in the Review and Flight datasets. This can be attributed to their higher interaction density (computed as #node degree / #time steps, as described in Line 1408), where nodes tend to feature a larger number of simultaneous interactions. Such patterns result in pronounced burstiness and stronger temporal clustering, thus enhancing the effectiveness of TG-Mixer.

We hope our revised paper has addressed your concerns.

Table 2: A subset of MRR results on TGB.

We sincerely appreciate your recognition of our model novelty, conducted experiments, and writing presentation, as well as your acknowledgment of the other aspects of our rebuttal.

Based on your expanded explanation for "popularity", we provide a more detailed response to W1 under the view of the Recommender System:

• Temporal clustering provides inductive bias or auxiliary shortcuts for capturing preferences of users, rather than simply recommending popular items. We understand your concern, which can be explained as: recommending a suitable item for a user should prioritize the relevant items that reflect its preference (e.g., clothing in your provided example), rather than only recent popular nodes (e.g., computers in your example). However, TG-Mixer has already considered both, as it constructs neighborhoods to summarize node preference (achieved by the token mixer in Line 292) and encodes temporal clustering information to amplify and mix the neighborhood information for representations (achieved by the information mixer in Line 303). Therefore, TG-Mixer may tend to recommend relevant and recent popular nodes to the user, thereby enhancing recommendation performance.
• Predicting new links may indeed pose challenges for TG-Mixer, which is a recognized potential limitation in Line 945. As discussed in Line 945, new links (e.g., a user who has never purchased electronics in your example) may conflict with the motivation behind temporal clustering, as the node pairs of these links lack recent historical interactions but still represent positive links. Consequently, in datasets with a significant proportion of new links (e.g., Coin in Table 7 of Line 956), TG-Mixer achieves performance that is competitive with the baselines, which could be a limitation of TG-Mixer. However, as highlighted in Line 951, we emphasize that TG-Mixer performs best in the majority of datasets (eleven in total), particularly those where simultaneous interactions are more prevalent.

Once again, thank you for your valuable insights and the recognition of our work. If you have any further questions or concerns, please do not hesitate to reach out; we would be more than happy to address them. :)

Thank you for considering our work! We are happy to provide further clarification based on your additional question.

As mentioned in Lines 912–914, for batch processing, we maintain the rhythm vector in a batch with dimensions corresponding to the batch size, which is dynamically updated during training. Simply setting the batch size to 1 could train the model where all nodes share the same rhythm state, as the interaction is inputted one by one. However, we did not conduct this experiment because setting the batch size to 1 would significantly slow down the training process --- at least 200 times slower than the current setup!

For 100 negative nodes, we modify get_link_prediction_metrics() in the DyGLib. For each batch (suppose the batch size is 200), we select the first 100 negative samples from 200 generated negative samples as the final negative samples for the MRR metric. We believe this method could achieve the purpose of the negative sampling strategy during the evaluation phase. The pseudocode is as follows:

1. y_pred_pos, y_pred_neg = predicts[:num_pos], predicts[num_pos:]

2. y_pred_neg = y_pred_neg[:100]

3. computing MRR...

We are deeply grateful for your invaluable comments, which have significantly improved our paper. Your comprehensive review has been incredibly insightful.

We are deeply grateful for your invaluable comments! We would like to provide further clarification as follows:

As mentioned in Lines 1009-1040, we have evaluated and discussed the model performance of TG-Mixer and existing TGNs under different batch sizes. Empirically, the performance of most existing TGNs is influenced to varying degrees by batch size. This is because all predictions within a given batch share the same model state and memory information, which may become outdated for later interactions in the batch. Consequently, the model training is more likely to experience greater information loss when using a larger batch size. For further details, please refer to Lines 1009-1040.

We would like to clarify that our computation of the MRR metric is reasonable and aligns with practices commonly used in recent works and their official implementations. We follow the TGB benchmark [1, 2] and reference their official implementations [url1, url2, url3] to evaluate model performance using MRR for temporal link prediction. Specifically, for each positive sample, we calculate its ranking among 100 negative samples to compute the MRR. For efficiency, all positive samples within a given batch share the same 100 negative samples. Therefore, in the provided pseudocode, for all positive samples in a batch, i.e., y_pred_pos, we sample 100 negative samples, i.e., y_pred_neg[:100]. We believe this computation is both efficient and reasonable because it satisfies the randomness requirement for negative sampling while avoiding the significant complexity that arises from computing 100 negative samples for every positive sample.

Thank you for your attention to our work; your feedback has been extremely helpful in improving our paper!

[1] DTGB: A Comprehensive Benchmark for Dynamic Text-Attributed Graphs, NeurIPS, 2024.
[2] Temporal Graph Benchmark for Machine Learning on Temporal Graphs, NeurIPS, 2023.
[url1] https://github.com/zjs123/DTGB/blob/main/evaluate_models_utils.py#L157
[url2] https://github.com/shenyangHuang/TGB/blob/main/tgb/linkproppred/evaluate.py#L76
[url3] https://github.com/shenyangHuang/TGB/blob/main/examples/linkproppred/tkgl-wikidata/tlogic.py#L75

We sincerely appreciate your thoughtful attention to our work. Regarding the concerns you raised, we would like to offer the following clarifications:

For Finding 1.

We believe that the conclusion of Finding 1 is reasonable and well-supported. Temporal differences across different batches (e.g., information from the previous batch) are addressed through the model update via Back Propagation Through Time (BPTT). On the other hand, the outdated information issue we discuss pertains to outdated information within the same batch. The fundamental reason for this problem is that all predictions within the batch share the same model state and the information for the later predictions is outdated. Therefore, we design distinct rhythm vectors to capture temporal differences among different predictions within a given batch. This approach prevents one shared rhythm vector for all predictions within a batch, thus effectively avoiding the outdated information issue from the later predictions of the batch.

For Finding 2.

We provide the detailed computation process for MRR with fixed source nodes as follows:

Step 1: Computing positive links probabilities. This process is consistent with our implementation. For clarity, we describe it as pos_probs = TGMixer(pos_src_nodes, pos_dst_nodes, interact_times).

Step 2: Computing negative links probabilities. We follow the implementation in [1] and sample 100 negative destination nodes for each batch. The source nodes within a batch share the same set of 100 negative destination nodes. The detailed implementation for each batch is as follows (Reference: [url1]):

---

1. Sample 100 negative destination nodes, resulting in neg_dst_nodes.
2. Initialize all_neg_probs = [].
3. For eachone_neg_dst_node in neg_dst_nodes:
   • Duplicate one_neg_dst_node batch_size times, resulting in one_neg_dst_nodes.
   • Compute the negative link probabilities using neg_probs = TGMixer(pos_src_nodes, one_neg_dst_nodes, interact_times).
   • Append neg_probs to all_neg_probs.

---

This process yields the negative link probabilities for all positive source nodes corresponding to the 100 negative destination nodes.

Step 3: Computing MRR. For each positive link probability in pos_probs, we retrieve the corresponding pos_src_node probabilities from all_neg_probs for the 100 negative links, and finally compute the MRR.

Finally, we acknowledge that MRR with fixed source nodes is more aligned with recommendation systems. However, we believe that the MRR results with non-fixed nodes presented in our paper at least provide a fair testbed for performance evaluation, as all models are evaluated using the same metric computation method. In future versions, we will clearly document our MRR computation methods and revise the results to include MRR with fixed source nodes. Thanks for your advice!

[1] DTGB: A Comprehensive Benchmark for Dynamic Text-Attributed Graphs, NeurIPS, 2024.
[url1] https://github.com/zjs123/DTGB/blob/main/evaluate_models_utils.py#L218

We sincerely appreciate your meticulous attention and valuable suggestions, which have significantly made our work more thorough and rigorous.

Based on your invaluable feedback, we conduct additional empirical evaluations that confirm the following two findings, and we will incorporate these discussions into the revised version of our paper.

Finding 1: The batch-based rhythm vector could serve as an effective technique to mitigate the outdated information issue in existing batch training approach.

As mentioned in Line 1019, we would like to clarify that TG-Mixer is a memory-based method because it requires maintaining an up-to-date rhythm vector component, which is updated at the batch level.

Empirical Analysis. As stated in Lines 1016–1018, a limitation of the existing training approach in TGNs is that all predictions within a given batch share the same model state, which may become outdated for later interactions in the batch. We believe that the batch-based rhythm vector offers an effective solution to this issue by maintaining separate rhythm vectors for each prediction within the same batch. This enables TG-Mixer to capture the temporal differences between interactions of a batch, thereby mitigating the adverse effects of outdated information during training.

Experimental Evaluation. To assess the effectiveness of the batch-based rhythm vector, we conduct an additional ablation study. Specifically, we use the same rhythm vector for all nodes within a batch and update it globally. This results in a new variant, and we call it "TG-Mixer$_{\text{w/o. batch rhythm}}$".
The experimental results are represented in Table 1, and we will include them in Table 5 of our paper. We observe a performance drop for TG-Mixer when all nodes within a batch share the same rhythm vector. This highlights the effectiveness of the batch-based rhythm vector, which successfully mitigates the outdated information issue inherent in the existing batch training approach.

Table 1: Transductive AP (%) results of ablation study.

Finding 2: TG-Mixer consistently demonstrates superior performance on the MRR metric when computed with fixed source nodes.

Empirical Analysis. In our implementation, we compute MRR by ranking a specific positive edge among 100 complete negative edges, where the negative source and destination nodes may differ from the positive ones. We believe this approach strikes a balance between fair evaluation and efficiency. On the other hand, in real-world applications such as recommendation systems, the effectiveness of recommending items for a given user is often of primary concern. Therefore, we conduct an additional evaluation using the MRR metric, where the source node (i.e., user) is fixed, and 100 negative edges are constructed by sampling 100 negative destination nodes (i.e., potential recommended items) for this source node.

Experimental Evaluation. The experimental results are presented in Table 2, and we plan to add a new section in our paper to incorporate this discussion accordingly.

TG-Mixer continues to achieve the best performance under the MRR metric when the source nodes are fixed, further demonstrating its effectiveness and superiority. Notably, when combined with the results reported in our paper, TG-Mixer achieves the best performance across AP, ROC-AUC, and the MRR metrics for fixed and non-fixed source nodes, highlighting its robustness and overall effectiveness.

We sincerely appreciate your meaningful feedback, which has been instrumental in improving our work. Thank you once again!

Table 2: Transductive MRR results with fixed source nodes.

Thank you for your attention. Based on your further question, we provide the following response:

We would like to emphasize that the single rhythm vector is updated recursively in a per-sample manner. This means that the single rhythm vector is progressively updated according to the position of predictions within the batch. For example, assuming the positions of the batch samples are $p_1, \dots, p_{200}$, the single rhythm vector is updated sequentially from position $p_1$ to $p_{200}$. Even though the interactions are already sorted by timestamp, this approach overlooks the fact that interactions often share a large number of identical timestamps. Assuming the interactions at positions $p_{100}$ and $p_{101}$ occur at the same timestamp, the single rhythm vector incorporates the information from the position $p_{100}$ to determine the state for $p_{101}$. This process introduces unintended temporal dependencies between these two simultaneous interactions.
To address this issue, the batch-based rhythm vector method assigns distinct rhythm vectors to different positions within the batch, allowing each rhythm vector to rely exclusively on the timestamp information of its corresponding position. This approach effectively eliminates the erroneous temporal dependencies introduced by per-sample updates, ensuring an up-to-date representation of the temporal information for each interaction.

Regarding the computation of MRR, we sincerely appreciate your valuable suggestions regarding efficiency. In future versions of the paper, we will revise our description of the MRR calculation and incorporate the experimental results for MRR with fixed source nodes.

Finally, we would like to express our heartfelt gratitude for your insightful comments, which have greatly contributed to making our paper more rigorous!

We would like to sincerely thank Reviewer qT7Y for providing a detailed review with insightful questions.

W1: Explanation of the conclusion for empirical analyses.

Quick Answer: We would like to clarify that our empirical analyses are fair because they are specifically designed to compare the differences between truly existing links and links that would never occur.

Our empirical analyses do not consider the randomly sampled interaction nodes as "potentially interacting pairs". Instead, they represent "the pairs that would never link" as you noted. As outlined in Lines 157-158, our empirical analyses compare truly existing interaction nodes with randomly sampled ones. This setup aligns directly with the link prediction task, where the goal is to distinguish positive links (those that truly exist) from negative links (those that do not). Therefore, the conclusions drawn from our empirical analyses are both reasonable and well-supported.

W2: Explanation of bursty and periodic interactions.

Quick Answer: We do not differentiate between bursty and periodic interactions in our analyses because both of them exhibit temporal clustering behavior.

Empirical Analysis. Both bursty and periodic interactions exhibit temporal clustering, with the key distinction being whether their timestamps follow a regular pattern.

• Bursty interactions are characterized by unevenly distributed timestamps, and typically demonstrate temporal clustering during specific periods.
• Periodic interactions, in contrast, feature regular timestamps but can also demonstrate temporal clustering at a specific timestamp, as a large number of interactions may occur simultaneously at this timestamp.

Since our focus is not on the regularity of timestamps, we do not separately analyze these two types of interactions.

Experimental Evaluation. As noted in Q4, we have incorporated periodic datasets into the model evaluation. TG-Mixer significantly outperforms the baseline models on these periodic datasets, further demonstrating its effectiveness. For additional details, please refer to Q4.

---

Thanks for your insightful review, which has guided our revisions! As revised in Lines 242-245, we elaborate on temporal clustering for temporal link prediction based on your insightful review in W2. Temporal clustering can be explained as follows:

• (i) If a given node has recent interactions, then it is more likely to interact at the current timestamp;
• (ii) If the node has only distant past interactions or even no historical interactions, it is less likely to interact now.

Q1-1: Comparison with existing burstiness study.

As revised in Lines 1306-1314, we have identified some existing studies on burstiness that explore the effects of bursty interaction patterns. However, these related works focus on physical science and outside the ML community, and we have not found relevant studies within the ML community.

Difference Comparison. Although there indeed exist some works that explore burstiness, these works focus on designing statistical models to analyze bursty events and their impacts on the distribution characteristics, correlations, and propagation processes. They are outside the ML community. Instead, we present the first attempt to investigate temporal clustering in temporal graphs, and incorporate this insight for advancing temporal link prediction.

Q1-2: Explanation of motivation for silence decay in temporal graphs.

Quick Answer: We would like to clarify that the silence decay mechanism may not be directly relevant to high-order neighbors in graph data but is highly relevant to first-hop neighbors in temporal graphs.

• We investigate the activity patterns in temporal graphs and study invaluable insights for temporal link prediction. As mentioned in Lines 246-251, the silence decay mechanism stems from the implications of empirical analyses. As shown in Figure 1 of Line 66, in temporal graphs, the first-hop neighbors of specific nodes can be represented as an interaction sequence, which reveals their activity patterns. As illustrated in Line 243, we study such patterns and look for potential insights for temporal link prediction.
• Motivation for silence decay. As detailed in Figure 3 of Line 181, inter-event times play a crucial role in incorporating the powerful node interaction rhythms. To this end, we designed the silence decay mechanism to reflect current rhythm patterns, promoting the inclusion of temporal clustering information for temporal link prediction.

We are sincerely grateful for your positive assessment! We are deeply grateful for your recognition and suggestions, and we have revised our paper based on your reviews.

Once again, many thanks for your insightful efforts. Your detailed suggestions help us a lot!

W3: Explanation of temporal link prediction.

Quick Answer: Several sections emphasize that the link prediction task is performed at a specific timestamp.

• Preliminaries. As shown in Lines 124 and 1290, temporal link prediction is a well-defined and widely studied task, which is conducted at a specific timestamp.
• Solution. Line 356 states that the Temporal Link Decoder module predicts link existence at a specific timestamp.
• Experimental evaluation. We follow the standard task and evaluation setting in [1]. As discussed in Line 917, we use the common setup and employ a mini-batch training with a batch size of 200. Although we conduct parallel training and inference in batches, node pairs and corresponding timestamps in each batch are completely independent. Thus, the batch size does not typically affect the performance.

[1] Towards Better Dynamic Graph Learning: New Architecture and Unified Library, NeurIPS, 2023.

W4-1: Evaluation under four negative sampling strategies.

Quick Answer: TG-Mixer demonstrates outstanding performance across four challenging negative sampling strategies under the MRR metric, reinforcing its effectiveness.

Revision: We have incorporated these discussions in Lines 972-1007.

Empirical Analysis. As stated in Line 908, We evaluate model performance using the random sampling strategy. Recently, some works [1, 2] have introduced advanced evaluation methods for temporal link prediction in temporal graphs. To fully assess the models, we adopt a more challenging evaluation with four negative sampling strategies and rank-based metrics.

Experimental Evaluation. Based on the above motivation, we conduct additional experiments to evaluate model performance under four different negative sampling strategies using the MRR metric. Their descriptions can be found in the Implementation Details.

Due to char limitations, we provide key findings and results here. Other findings can be found in Lines 972-1007.

• TG-Mixer consistently performs best across four negative sampling strategies under the MRR metric, further proving its robustness.
• Inductive negative sampling often leads to the most performance drop. This is likely because this strategy samples unseen nodes to construct negative links, making it more challenging for models to distinguish between positive and negative links.

Table 1: A subset of transductive MRR results on Reddit.

Implementation Details. In addition to random negative sampling, we follow [1] and use historical negative sampling (sampling negative links that have been observed) and inductive negative sampling (sampling negative links that are unseen during training). To mitigate potential biases from popular nodes, we conduct degree-aware negative sampling by sampling negative nodes with the probability of $d/N$, where $d$ is the node's current degree and $N$ is # historical interactions. We follow [2] and sample 100 negative edges per positive edge and employ MRR as the metric.

[1] Towards Better Evaluation for Dynamic Link Prediction, NeurIPS, 2022.
[2] Temporal Graph Benchmark for Machine Learning on Temporal Graphs, NeurIPS, 2023.

W4-2: Evaluation on additional datasets in TGB benchmark.

Quick Answer: TG-Mixer exhibits superior performance across the additional datasets in the TGB benchmark.

Empirical Analysis. In the original paper, we conduct experiments on the two selected datasets because many baselines may not be implemented on other datasets due to computational issues ('NA' in the following Table). To further validate the model performance, as revised in Line 956, we add three datasets: Review, Comment, and Flight.

Experimental Evaluation. We provide a subset of experimental results here. For the complete results, please refer to Table 7 of Line 956.

• TG-Mixer consistently demonstrates superior performance on the additional datasets, highlighting its robustness and strong capabilities.
• The most significant performance improvements are observed in the Review and Flight datasets. This can be attributed to their higher interaction density (computed as #node degree / #time steps, as described in Line 1408), where nodes tend to feature a larger number of simultaneous interactions. Such interaction patterns result in pronounced burstiness and stronger temporal clustering, thus enhancing the effectiveness of TG-Mixer.

We hope our revised paper has addressed your concerns!

Table 2: A subset of MRR results on TGB.

We sincerely thank Reviewer nfv5 for providing the insightful review, and we have revised our paper based on your review carefully.

W1-1: Difference discussion and contribution reaffirmation.

Quick Answer: Unlike the existing burstiness studies in the physical science community, we present the first model that incorporates burstiness for temporal graph representation learning.

Revision: For readability, we have updated the term from "silence duration" to "inter-event times" throughout the paper and developed a discussion on existing burstiness studies in Lines 1303–1311.

Existing Burstiness Studies. As revised in Line 1301, although some works explore burstiness in sequential data, they focus on designing statistical models to analyze bursty events and their impacts on distribution, correlations, and propagation. These are outside the ML community.

Our Work. Instead, we leverage burstiness to design a deep-learning model tailored for representation learning in temporal graphs, and we highlight our key contributions:

• Empirical analyses for temporal clustering. We introduce statistical empirical analyses and present the first attempt to identify temporal clustering in temporal graphs, i.e., the activity of certain nodes exhibits burstiness.
• A novel model for temporal link prediction. Building on the insights from empirical analyses, we propose the first model that explicitly captures burstiness information for temporal link prediction.

W1-2: Explanation of mean operation in empirical analyses.

Quick Answer: We would like to clarify that the mean operation is statistically valid, and the alignment between our findings and the conclusions in existing studies further supports its reliability.

• Mean operation is also utilized in the analysis of existing studies on burstiness. Based on the existing studies [1, 2, 3, 4], the inter-event times for specific nodes or timestamps exhibit a heavy-tailed distribution. The mean operation in empirical analyses derives the inter-event times for each node or timestamp, which is also conducted in existing studies. For example, in Eq. 1 of [1] and Fig. 1 of [2], the authors compute the average inter-event times for each node to model the heavy-tailed distribution.

• Conclusion reliability. As shown in Figure 3, the inter-event times for truly existing interaction nodes follow a typical heavy-tailed distribution. This finding is consistent with the conclusion in existing studies [1, 2, 3, 4], which further supports the validity of analyses.

[1] Modeling temporal networks with bursty activity patterns of nodes and links, Phys Rev. Research 2, 2020.
[2] Small But Slow World: How Network Topology and Burstiness Slow Down Spreading, PRE, 2011.
[3] Constructing temporal networks with bursty activity patterns, Nature Communications, 2023.
[4] Copula-based Analysis of the Autocorrelation Function for Simple Temporal Networks, KPS, 2023.

W2-1: Explanation of the rhythm vector.

Revision: In Lines 285-347, we have revised and rewritten the description of the TG-Mixer encoder.

Motivation Analysis. As mentioned in Lines 303-310, we introduce the rhythm vector $\mathbf{C}_\text{rhythm}^t$ to encapsulate the condensed essence of interaction rhythms at each timestamp. This vector is updated globally and chronologically through the temporal mixer. Intuitively, the interaction rhythm for the next timestamp is influenced not only by the current rhythm patterns (achieved by the silence decay mechanism) but also by the historical rhythms among recent interactions (achieved by the information mixer).

Update Process. Based on this motivation, as clearly shown in the "Temporal Mixer" part of Figure 4, the update process of the rhythm vector consists of two steps: (i) a silence decay mechanism that captures current rhythms through inter-event times (Eq. 3, 4), and (ii) the information mixer, which updates the rhythm vector with historical rhythms among recent interactions (Eq. 5).

W2-2: Motivation of the $\mathbf{C}_ {\text{temp}}^t$ and $\mathbf{C}_ {\text{decay}}^t$.

Revision: In Lines 311-323, we have revised the description of the silence decay mechanism.

As noted in Line 311, the silence decay mechanism aims to update the rhythm vector with current rhythm patterns. In Line 320, we have mentioned that the rhythm vector records the long-term state because it is frequently updated across timestamps. Therefore, in our silence decay mechanism, we first generate short-term rhythm state $\mathbf{C}_ {\text{temp}}^t$ using a neural network. Then, the long-term rhythms are decayed based on the inter-event times, thus capturing and encoding temporal clustering patterns among current rhythms. This results in the decayed vector $\mathbf{C}_ {\text{decay}}^t$, which contains the current rhythm pattern information. More details can be found in Lines 311-323 of our revised paper.

Thank you for your thoughtful review and for recognizing the innovation, model design, extensive experiments, and exceptional effectiveness of TG-Mixer, as well as for acknowledging other parts of our rebuttal.

We are also deeply grateful for your meticulous comments, which have helped us identify an issue within the community.

Task definition on the batch scale.

We carefully reconsider the widely used task definition in the temporal graph community, which is typically framed at individual node pairs. In practice, however, most methods employ a batch-based training approach to enable parallel processing. Therefore, as revised in Lines 132-135, we define our task on the batch scale.

Definition 2. Temporal Link Prediction. Given a batch size $B \in \mathbb{Z}^+$, a set of interaction nodes \left\\{u_b \in \mathcal{V} \right\\}_ {b=1}^B and \left\\{v_b \in \mathcal{V}\right\\}_ {b=1}^B, and corresponding timestamps \left\\{t_b > 0 \right\\}_ {b=1}^B, temporal link prediction aims to predict whether each node pair $\left(u_b, v_b\right)$ interacts at timestamp $t_b$ based on the historical links \left\\{(u', v', t') | t' < t_b\right\\} \subseteq \mathcal{G}, i.e., predicting the existence of the future link $(u_b, v_b, t_b)$.

Explanation of batch-based training approach.

Quick Answer: Memory-based TGNs tend to face challenges with large batch sizes due to the outdated memory state within the batch.

Revision: We have incorporated these discussions in Lines 1009-1040.

Empirical Analysis. The batch-based training approach, introduced by [1], has been widely adopted in the temporal graph community. This method first sorts all interactions by timestamp and then groups them into batches using the predefined batch size. In this way, Back Propagation Through Time (BPTT) within each batch allows the models to update in chronological order.
Despite its efficiency, a fundamental issue with this approach is that all predictions within a given batch share the same model state, which may be outdated for later interactions in the batch. It is particularly problematic for memory-based methods, as they always explicitly maintain an up-to-date memory component, such as the rhythm vector in TG-Mixer and node state in TGN [1]. While the memory state for the first interaction in the batch is up-to-date (as it incorporates information from all previous interactions), the memory state for later interactions in the batch is out-of-date (as it does not include information from previous interactions within the same batch). Consequently, a larger batch size may result in more information loss.

Experimental Evaluation. Based on the above motivation, we conduct additional experiments to evaluate model performance under different batch sizes.

From the results, we find that:

• TG-Mixer continues to achieve the best performance across various batch sizes, further validating its effectiveness and robustness.
• Although the performance ranking of models remains unchanged, memory-based methods (e.g., TG-Mixer and TGN) tend to perform better with smaller batch sizes. This underscores the limitations of the existing batch training approach when applied to larger batch sizes. It also highlights the need for a more effective parallel processing method, which is beyond the scope of this work and we leave this as a future research direction.

Once again, thanks for your valuable suggestions during both the review and discussion phases.

Table 1: MRR results under different batch sizes on Wikipedia. "*" denotes memory-based methods.

[1] Temporal Graph Networks for Deep Learning on Dynamic Graphs, ICLR, 2020.

We would like to sincerely thank Reviewer S1Kp for providing a detailed review with insightful questions.

W1: Reaffirmation of novelty and contribution.

Quick Answer: We present the first attempt to investigate temporal clustering in temporal graphs and successfully incorporate this insight for advancing temporal link prediction.

TG-Mixer extracts neighborhood information through a standard technique used in existing TGNs, but it is not the main focus of our work. As noted in Lines 75-107, we highlight our key contributions as follows:

• Empirical analyses for temporal clustering. We introduce statistical empirical analyses and present the first attempt to identify temporal clustering in temporal graphs, i.e., the activity of certain nodes exhibits burstiness.
• A novel model for temporal link prediction. Building on the observations from empirical analyses, we re-think the interaction dynamics and propose a novel model that explicitly captures burstiness information for temporal link prediction.

W2 (Q1-1): Motivation and update process of $\mathbf{C}_ \text{rhythm}^t$.

Revision: As shown in Lines 302-347 and 912-915, we have revised the paper to provide more details of the rhythm vector.

Motivation Analysis. As mentioned in Lines 303-310, we introduce the rhythm vector $\mathbf{C}_\text{rhythm}^t$ to encapsulate the condensed essence of interaction rhythms at each timestamp. This vector is updated globally and chronologically through the temporal mixer. Intuitively, the interaction rhythm for the next timestamp is influenced not only by the current rhythm patterns (achieved by the silence decay mechanism) but also by the historical rhythms among recent interactions (achieved by the information mixer).

Update Process. Based on this motivation, as clearly shown in the "Temporal Mixer" part of Figure 4, the update process of the rhythm vector consists of two steps: (i) a silence decay mechanism that captures current rhythms through inter-event times (Eq. 3, 4), and (ii) the information mixer, which updates the rhythm vector with historical rhythms among recent interactions (Eq. 5). As revised in Lines 912-915, for batch processing, we maintain a matrix with a size corresponding to the batch size, which is dynamically updated throughout training.

Q1-2: Handling simultaneous interactions.

Quick Answer: We would like to clarify that as noted in Lines 1407–1414, we have already discussed this situation where TG-Mixer handles a large number of simultaneous interactions.

Empirical Analysis. We introduce interaction density to quantify simultaneous interactions among datasets. As mentioned in Line 1408 of Section D.4, we define interaction density (the averaged node degree at each timestamp) to quantify the degree of simultaneous interactions. When there is a large number of simultaneous interactions, it reflects high interaction density and more pronounced burstiness in nodes' activity, thus amplifying temporal clustering. Therefore, as demonstrated in Figure 6 in Line 448, the decay coefficient in the update process of the rhythm vector (Eq. 4) can leverage the shorter inter-event times to provide more distinguishable signals, leading to improved model performance.

Experimental Evaluation. The results in Line 324 of Table 1 show that TG-Mixer significantly outperforms baselines on datasets with more simultaneous interactions, particularly on the US Legis. and Flights datasets. This may be because baselines struggle to provide differentiated encoding when multiple interactions share the same timestamp. In contrast, TG-Mixer effectively captures high-level temporal clustering, making it less susceptible to this issue. Further details can be found in Section D.4.

Q2: Discussion between TG-Mixer and GraphMixer.

First of all, we would like to clarify that we did not state that GraphMixer samples a "massive number of historical neighbors". We referenced GraphMixer in this context to highlight the limitations of existing works, as these limitations are also supported by the GraphMixer.

Kindly note that our novelty and contribution extend far beyond the neighbor selection strategy (see W1 for further details). While the neighbor selection strategy is technically similar to that of GraphMixer, the key difference lies in the motivation:

• GraphMixer samples recent historical links with the goal of "avoiding poor model trainability and generalization ability".
• In contrast, as mentioned in Lines 246-251, the neighbor selection strategy in TG-Mixer is driven by empirical analyses, where we believe that the latest rhythm patterns are critical for predicting future links.

Thank you once again for your constructive comments and for adjusting your contribution score during the discussion phase! Your positive feedback, particularly regarding our novelty, contribution, and presentation, has been incredibly inspiring to us. We are proud to present the first attempt at investigating temporal clustering in temporal graphs and leveraging this insight for temporal link prediction. Your detailed suggestions have been immensely valuable in enhancing our work.

Once again, thank you for your dedicated efforts and the positive feedback!