Recent Link Classification on Temporal Graphs Using Profile Builder

Thank you for your thorough review, and your specific feedback. We are developing a plan to address the weaknesses that you have identified, and we would like to confirm with you that this plan would address your concerns.

• W1: We will add a simple illustrative example indicating the differences between FLP, RLC, and DNC.
• W1: The graphs that are currently considered are all bipartite and heterogeneous with interactions typically being between a user → item. This is not required for the RLC task in general, but is the structure of our dataset under consideration. In such a setting, we would argue that the metapath construction is a relatively natural fit.
• W1: Metric uncertainty - We will add an illustrative example that highlights situations where MCC can correctly handle label imbalance but more traditional metrics cannot. In addition, we will leverage the edge homophily definition to explain aspects of our results, in analogy to the discussion of node homophily for node classification.
• W2: We will rework the problem statement to address the lack of clarity. Specifically, we will make clear that we are working in a continuous time setting, the source of our edge features, and the transductive nature of the problem.
• W2: We will clarify the definition of a profile in this setting, rather than depending on the colloquial definition.
• W2: The graphs in the figures are illustrative but they are dynamic based on the content of the containers. We will make this more clear in the text and figure captions
• W3: We will add two additional datasets, one based on video game reviews and one on NFT sales, and baselines from GraphMixer and TGAT. The NFT sales dataset is considerably larger than the ones in the current work.
• W4: We will rework the text to highlight the importance of our new task and more clearly explain how it differs from FLP and DNC; and we will use that to further motivate our discussion about the value of this new task. We hope that by addressing the metrics as we have planned in W1, and clearly explaining the value of RLC, we can address W4.
• W5: We thank the reviewer for pointing out typos and other places where the work lacks clarity. We will address those, and carefully edit the paper to catch others.

We aim to have this work completed before the end of the review process, so that we can address any additional concerns that you may have. Please do let us know if this plan of action is insufficient to address your points.

Best Regards,

The Authors

We again thank the reviewer for their thorough review and specific feedback. We have just uploaded a new draft of the paper that has addressed the points that have been raised. Specifically we have:

• W1: We have added such a figure as figure 1.
• W1: We have added an illustrative example of a situation where MCC and APS give conflicting results, and provided an explicit citation to a paper that provides a clear argument for MCC over AUC.
• W2: We have updated the problem definition provided in Section 3 in the light of your questions.
• W3: We have added two additional datasets and two additional baselines in Table 3.
• W4: We have made the difference between RLC, FLP, and DNC much more clear.
• W5: We have given the paper a thorough edit, and we will perform one more pass of editing on Monday to catch any last typos.

We welcome any further questions. Please don’t hesitate to share them so that we can address them as quickly as possible.

Best Regards, The Authors

Thank you for your thorough review, and your specific feedback. We are developing a plan to address the weaknesses that you have identified, and we would like to confirm with you that this plan would address your concerns.

• Our Graph Profiler method is novel in so far as, to our knowledge, the concept of building time-dependent profiles for entities based on based on the features and labels of their previous interactions is not something we are aware of. In contrast, TGN learns a graph wide memory. We will update the text to make this point clear. Also, we believe that we have additional contributions, including the introduction of a new research task, an associated figure of merit, and baseline results. We will rework the text to further emphasize these contributions.
• On the topic of extensive evaluations being required, we will add two additional datasets, one based on video game reviews and one on NFT sales, and baselines from GraphMixer and TGAT. The NFT sales dataset is considerably larger than the ones in the current work.
• On the topic of critical design principles, we will rework the text to emphasize that these principles comprise the design space for a family of models.
• We will include our investigations of the identified design space for the FLP in an appendix and make reference to it in the main text.

We aim to have this work completed before the end of the review process, so that we can address any additional concerns that you may have. Please do let us know if this plan of action is insufficient to address your points.

Best Regards,

The Authors

We again thank the reviewer for their thorough review and specific feedback. We have just uploaded a new draft of the paper that has addressed the points that have been raised. Specifically we have:

• We have cleaned up the text and notation, taking the time to be more clear about many of our definitions.
• We have clarified the importance of the TGN modifications for motivating our design choices for our Graph Profiler model. This clarification can be found in the first paragraph of section 5.2. In summary, we explored modifications to TGN that borrowed specifics from other SOTA models, such as the fixed time encoder, to understand the importance of these model-features for our RLC task. Our development of Graph Profiler was informed by the important components of our modified TGN.
• We have performed a related analysis to the above that explores the difference between RLC and FLP for a our modified TGN, and report this comparison in Appendix G.
• We have included time dependent measures of our edge homophily in Appendix A. We hope that this provides a clearer perspective on the way that our datasets evolve with time.
• We have added two additional datasets and two additional baselines in Table 3.
• We have given the paper a thorough edit, and we will perform one more pass of editing on Monday to catch any last typos.

We welcome any further questions. Please don’t hesitate to share them so that we can address them as quickly as possible.

Best Regards, The Authors

Thank you for your thorough review, and your specific feedback. We are developing a plan to address the weaknesses that you have identified, and we would like to confirm with you that this plan would address your concerns.

• Our Graph Profiler method is novel in so far as, to our knowledge, the concept of building time-dependent profiles for entities based on based on the features and labels of their previous interactions is not something we are aware of. Also, we believe that we have additional contributions, including the introduction of a new research task, an associated figure of merit, and baseline results. We will rework the text to further emphasize these contributions.
• We will address the typos through a thorough editing process.
• We will add an illustrative example of a situation where MCC and AUC give conflicting results and provide a clear argument for MCC.
• We will add more information about the time-encoder, and more generally we thank the reviewer for pointing this areas where the methods discussion is unclear, and we will work to improve that.
• We will add 2 additional datasets and additional benchmark results to make the analysis a bit more involved.

Additionally, we would appreciate if you could expand a little further on your question: “Is there any specific design of the metric for temporal interaction classification?” We are planning to answer it by expanding the MCC definition to explain that it applies in multi-class and imbalanced settings, but we are uncertain if this is a satisfactory answer for your question

We aim to have this work completed before the end of the review process, so that we can address any additional concerns that you may have. Please do let us know if this plan of action is insufficient to address your points.

Best Regards,

The Authors

We again thank the reviewer for their thorough review and specific feedback. We have just uploaded a new draft of the paper that has addressed the points that have been raised. Specifically we have:

• We have cleaned up the text and notation, taking the time to be more clear about many of our definitions.
• We have added an illustrative example of a situation where MCC and APS give conflicting results, and provided an explicit citation to a paper that provides a clear argument for MCC over AUC.
• We have added more information about the time encoder.
• We have included time dependent measures of our edge homophily in Appendix A. We hope that this provides a clearer perspective on the way that our datasets evolve with time.
• We have added two additional datasets and two additional baselines in Table 3.
• We have given the paper a thorough edit, and we will perform one more pass of editing on Monday to catch any last typos.

We welcome any further questions. Please don’t hesitate to share them so that we can address them as quickly as possible.

Best Regards, The Authors

Thank you for your thorough review, and your specific feedback. We are developing a plan to address the weaknesses that you have identified, and we would like to confirm with you that this plan would address your concerns.

• We thank the reviewer for pointing this areas where the methods discussion is unclear, and we will work to improve that. We will do so specifically by expanding upon details of our model, dimensions of matrices, and other model details; as well as the way in which we’ve constructed the dataset.
• We will expand our definition of edge homophily to be easier to understand.
• We will add 2 additional datasets and additional benchmark results to make the analysis a bit more involved.
• Knowledge Graph Link Prediction (KGLP) is a related problem but is different for multiple reasons. The first is that in RLC, the prediction is performed conditioned on the existence of the link. The second is that KGLP implies an ontological structure to the relationships which can naturally be leveraged.
• As for the question of additional metrics, we will add plots of the edge-homophily over time and a metric that we are referring to as balanced edge homophily.
• We will address the typos through a thorough editing process.
• We will review the rest of the paper to ensure that formulas like that for $d_1$ are motivated and clear.

We aim to have this work completed before the end of the review process, so that we can address any additional concerns that you may have. Please do let us know if this plan of action is insufficient to address your points.

Best Regards,

The Authors

We again thank the reviewer for their thorough review and specific feedback. We have just uploaded a new draft of the paper that has addressed the points that have been raised. Specifically we have:

• We have cleaned up the text and notation, taking the time to be more clear about many of our definitions.
• We have clarified the definition of edge homophily by adding a paragraph that discusses its intuitive definition.
• We have included time dependent measures of our edge homophily in Appendix A. We hope that this provides a clearer perspective on the way that our datasets evolve with time.
• We have added two additional datasets and two additional baselines in Table 3.
• We have given the paper a thorough edit, and we will perform one more pass of editing on Monday to catch any last typos.
• We have clarified the importance of the TGN modifications for motivating our design choices for our Graph Profiler model. This clarification can be found in the first paragraph of section 5.2.
• We thank the reviewer for directing us to the link between RLC and Dynamic KG link prediction These tasks are quite related, but our current formulation of RLC has the additional complexity of delayed label observations. We have formulated this task in this way to match common industrial tasks such as fraud or toxicity detection.

We welcome any further questions. Please don’t hesitate to share them so that we can address them as quickly as possible.

Best Regards, The Authors

Thank you for your thorough review, and your specific feedback. We are developing a plan to address the weaknesses that you have identified, and we would like to confirm with you that this plan would address your concerns.

• We will work to understand the structure of the datasets under investigation and how their structure impacts the time encoding.
• To tease out the importance of the destination encoder, we will perform an ablation study and report those results. In addition to the destination encoder ablation study, we will perform other ablation studies and results in the main text of the paper.
• We will explore the link between RLC and dynamic KG relationship classification. We thank the reviewer for pointing this connection out as the authors are unfamiliar with this literature.
• Our Graph Profiler method is novel in so far as, to our knowledge, the concept of building time-dependent profiles for entities based on the features and labels of their previous interactions is not something we are aware of. Also, we believe that we have additional contributions, including the introduction of a new research task, an associated figure of merit, and baseline results. We will rework the text to further emphasize these contributions.
• To address the figure captions, we will remake the figures such that they are easier to read.
• We will address the typos through a thorough editing process.
• We will add 2 additional datasets and additional benchmark results.

We aim to have this work completed before the end of the review process, so that we can address any additional concerns that you may have. Please do let us know if this plan of action is insufficient to address your points.

Best Regards,

The Authors

We again thank the reviewer for their thorough review and specific feedback. We have just uploaded a new draft of the paper that has addressed the points that have been raised. Specifically we have:

• Performed an ablation study to understand the role of destination embeddings, which can be found in Appendix C. We observe that in 5/6 datasets, the destination embeddings help, and we present an explanation for why this would be true.
• We have explored the structure of the datasets and unfortunately have no explanation for the importance of the time encodings in the TGN framework. We do want to make clear, however, that the Graph Profiler model only ever uses learnable time encodings.
• We have added two additional datasets and two additional baselines in Table 3.
• We have given the paper a thorough edit, and we will perform one more pass of editing on Monday to catch any last typos.
• We thank the reviewer for directing us to the link between RLC and Dynamic KG relationship classification. These tasks are quite related, but our current formulation of RLC has the additional complexity of delayed label observations. We have formulated this task in this way to match common industrial tasks such as fraud or toxicity detection.

We welcome any further questions. Please don’t hesitate to share them so that we can address them as quickly as possible.

Best Regards, The Authors