Enhancing Temporal Knowledge Graph Completion with Global Similarity and Weighted Sampling

We appreciate the reviewer's time and insightful feedback.

1. Real-world deployment considerations are not discussed. For example, analyzing memory/compute needs and ability to handle streaming data.

That is a great point. We have now added a table that report the memory and time complexity of each step of our approach both theoretically and in dataset ICEWS14. The table is copied below for ease of access.

Notation: $b$, $n$ and $d$ represent the batch size, number of similar entities, and the entity embedding dimensions. $|\mathcal{Q}|$ and $|\mathcal{E}|$ represent the number of entities and quadruples in the dataset respectively.

While weighted sampling is an intuitive idea, the sampling function used could be further explored and justified. For example, analyzing alternative frequency-based formulas or evaluating sampling directly from a learned importance weighting.

We thank the reviewer for this comment. We conducted experiments with various strategies, including mean, max, and min inverse frequencies of head and tail entities. During our hyperparameter selection process, the approach we ultimately chose was the one that yielded the best performance. In terms of exploring a learned importance weighting for each entity, we agree that this is a promising direction. We have mentioned this point in the future work section.

In Figure 2.a, it seems that the proposed method (Full) is outperformed by the proposed method (Enhancement Layer) on HITS@10, and the proposed method (Full) is outperformed by the proposed method (Weighted Sampling), any insights on that?

This is a great question. Our findings suggest that the enhancement layer improves performance for edges involving low and medium-degree entities, which aligns with its intended function. However, for edges involving higher-degree entities, the full methodology outperforms the individual components.

We appreciate the reviewer's time and insightful feedback. The reviewer has thoughtfully highlighted important considerations regarding the datasets:

Q1. Have you tried experimenting on more TKGs, such as WIKIDATA and GDELT?

While we agree that applying our approach to additional datasets would indeed be interesting, we were unfortunately unable to do so within the timeframe of this revision. However, it is important to note the inherent diversity of the two datasets currently included in our manuscript. Although ICEWS14 and ICEWS18 originate from the same ICEWS dataset, they are recognized as distinct benchmarks in the literature due to their differing distributions and densities. For example, ICEWS18 has 10 times more quadruples compared to ICEWS14 despite both of them covering a similar time duration within their respective years (see table below). The section 5.1 of our paper describes the dataset construction. We have now also revised this section to explain the dataset construction in more detail. The following table summarizes the statistics across these datasets:

Q2. Have you tried more TKGC methods? For example, Cygnet, LCGE? We are now adding another state-of-the-art model TANGO [1] to the paper. Preliminary results show that even the addition of our enhancement layer alone improves the performance. The results are provided in the following table:

Further details about how the enhancement layer was incorporated to TANGO is provided in our paper.

References

[1] Han, Zhen, et al. "Learning neural ordinary equations for forecasting future links on temporal knowledge graphs." Proceedings of the 2021 Conference on empirical methods in natural language processing. 2021. (Code)

We thank the reviewer for their time and their valuable feedback.

1. The motivation is not well established. It seems that the authors combine the two methods (global similarity and weighted sampling).

We have modified the paper to better explain the motivation for our approach. Briefly, the motivation for integrating these two methods stems from their complementary strengths in addressing the challenges of Temporal Knowledge Graph (TKG) completion, including sparse or non-existent local connections and training biases resulting from unbalanced distribution. The enhancement layer addresses the issue of sparse or non-existent local connections for infrequent (long-tail) entities, an area where traditional TKG completion methods, which primarily relying on local neighborhood proximity for entity representation, fall short. On the other hand, weighted sampling ensures that long-tail entities are not overlooked by the model during training. Our ablation study, detailed in Section X of the paper, demonstrates how each component contributes to addressing these challenges. Figure 2.a illustrates the effectiveness of the enhancement layer for continual learning. As depicted in Figure 2.b, it also contributes to improving performance for entities with mid-degree nodes. The full method enhances hit@1 for low-degree nodes.

2. In Table 1, many various recent works should be considered and discussed: Xu et al., 2023. Temporal knowledge graph reasoning with historical contrastive learning. Zhang et al., 2023. Learning Long- and Short-term Representations for Temporal Knowledge Graph Reasoning. Zhu et al., 2021. Learning from History: Modeling Temporal Knowledge Graphs with Sequential Copy-Generation Networks.

We appreciate the reviewer's suggestion regarding the related work. To clarify, our experiments were designed to demonstrate the effectiveness of our framework and its ability to be used on top of any state-of-the-art TKG model. In Table 1, we compare the performance of a model before and after the application of our framework. We have now added another state-of-the-art model TANGO [1] to the paper. Preliminary results show that even the addition of our enhancement layer alone improves the performance.

The results are provided in the following table:

References

[1] Han, Zhen, et al. "Learning neural ordinary equations for forecasting future links on temporal knowledge graphs." Proceedings of the 2021 Conference on empirical methods in natural language processing. 2021. (Code)

2. It would be useful if the authors could add other state of the art models (from their related work) to the comparison models. The experiments would be more convincing and would make a stronger point if the authors could show how the proposed methodology can be added in other models too, and to be able to see the performance of other models with the proposed extensions.

We thank the reviewer for their suggestion. We are now adding another state-of-the-art model TANGO [1] to the paper. Preliminary results show that even the addition of our enhancement layer alone improves the performance. The results are provided in the following table:

To concisely illustrate the integration of the enhancement layer in TANGO, which comprises two graph convolutional layers, we explored two approaches: (1) positioning the enhancement layer above the two convolutional layers, and (2) placing it between them. Our findings indicate that situating the enhancement layer atop the convolutional layers results in superior performance. Further details about this implementation have been elaborated in our paper.

3. Another useful aspect on the proposed methodology is to add the run times and the time complexities when the layer and the sampling strategy are added.

This is a great suggestion. We have added a table that reports the runtime and memory complexity of different variations of our model (weighted sampling, enhancement layer, full) in comparison with the naive Titer, as well as the actual per-epoch and total runtime of our method on the ICEWS14 dataset. The table is copied below for ease of access.

Notation: $b$, $n$ and $d$ represent the batch size, number of similar entities, and the entity embedding dimensions. $|\mathcal{Q}|$ and $|\mathcal{E}|$ represent the number of entities and quadruples in the dataset respectively.

Minor comment: In section 5.2, 3rd sentence, the citation was not added/compiled properly on LaTeX.

We thank the reviewer for their comment. We have edited the paper and revised the issue.

References

[1] Han, Zhen, et al. "Learning neural ordinary equations for forecasting future links on temporal knowledge graphs." Proceedings of the 2021 conference on empirical methods in natural language processing. 2021. (Code)