Deep graph kernel point processes

We thank the reviewer for the positive comments and confirmation of our experiments.

Regarding weakness, we have now provided a comprehensive comparison with related works in the literature, in Table A.1, and have explained the contribution/novelty of our paper in detail in Appendix A.

We want to point out that the extension to the graph kernel is non-trivial, and a lot of modeling and experiment validation development needs to be done. We have considered various types of graph kernels based on various GNNs and also consider the case when the graph topology is known, and when the graph topology is unknown, then we use the GAT. We provided extensive numerical experiments and compared them with multiple baselines and SOTA. In particular, since point process data over graph is so widely available, and there is still a lack of principled graph kernel design, we believe this work will be a valuable addition to the literature.

In our paper, in our log-barrier function equation in the appendix (E.1) and (E.2), we used a small constant b<0 inside the log-barrier, which means that we can allow the intensity to have small non-negativity. However, we do this to get an easy initialization: we can start with a reasonable guess that allows for a small amount of negativity and tune b to diminish to 0 as the training process continues, and in the end, we observe in the experiment in the end the learned intensity function induced by the log-barrier is always non-negative.

Indeed, the reference (Omi et al. 2019) did not consider marks, and their model can only be used for temporal point process. In order to compare with this method for the graph setting, we further extend their method to model the mark distribution on top of FullyNN. We have now provided complete details of the extended FullyNN in Appendix H.2.

Thank you very much. We have fixed these typos in the reference.

We now have discussed the potential limitation in the Conclusion of the paper. As we have explained to the above reviewer, current kernel representation still assumes that the influence over events is "additive," which is common in the kernel-based type of methods for point processes. To be more general, one can potentially adopt a non-additive model over events that can be more complex, and we leave it for future work.

We thank the reviewer for the positive comments. We now have clarified in Appendix A that incorporating GNN in the graph-based influence kernel is crucial and non-trivial, requiring a carefully designed model. Our paper is the first one to incorporate GNN in the influence kernel without any parametric constraints.

We thank the reviewer for the comment. We have clearly summarized the novelty of the method, which is not simply an addition of existing methods in fact, there are non-trivial challenges to overcome and carefully design the model considering the graph structure of the data in a principled manner while maintaining the statistical/stochastical law of the point process, which is achieved through the kernel representation. Moreover, as also mentioned by the other reviewer, the core novelty is "The idea of introducing graph neural networks to model the kernel of point processes is novel." "The idea of modeling kernels in point processes by GNNs is novel and meaningful to the development of the community." We will also provide answers to the remaining questions.

The proposed approach appears to be a combination of existing methods, which raises questions about its novelty.

We have now provided a comprehensive comparison with related works in the literature in Table A.1, and have explained the contribution/novelty of our paper in detail in Appendix A.

The paper needs to provide a detailed explanation of how edges between nodes are established.

We would clarify that for real-world datasets, we adopt GAT in our framework to infer the underlying graph structure and node interactions based on the learned attention weights from data, since the underlying graph structure is unobserved in real-world datasets and we have no prior knowledge about the graphs. The latent graph structure is constructed upfront in synthetic datasets.

Is there a theoretical basis for the superior performance against the THP-S and SHAP-G baselines? These baselines employ powerful transformers and graph attention mechanisms.

We first would point out that the model performance of neural point processes is affected by various factors (e.g., deep neural network architectures, model formulation of the point process, specific problems), and so far, a general theoretical basis can hardly be established.

Meanwhile, we would clarify there is no GNN architecture incorporated in THP-S and SHAP-G, and they model the conditional intensity function instead of the influence kernel, which may not be able to capture the repeated influence patterns effectively like our method.

The observed improvement in likelihood compared to the baseline is very minimal and possibly not statistically significant.

We clarify that we not only report the average metrics from multiple independent runs but also the standard errors of those metrics. We believe that the improvement is significant both from statistical and practical perspectives.

Clarify how this probabilistic prediction model is employed for predictions – whether it is based on random sampling or mean predictions.

The predictability of the proposed model is evaluated through sampling. As we clarify in the paper, we use the learned model on the training set to generate new event sequences over the entire time horizon, and compare the generated sequences with the testing sequences, in terms of the predicted event frequency and the predicted distribution of event types. The time MAE is calculated as the mean absolute error between the temporal frequency of generated events and testing events.

Table 3 indicates that the advantage of the graph-based approach diminishes as the number of nodes increases, which runs counter to my expectation.

We would point out that the comparison of the absolute difference of performance metrics across different point process datasets has little practical meaning, for they have diverse problem settings and data features. More importantly, our proposed method enjoys superior performance advantages against baselines consistently across different datasets.

Lack of crucial baseline comparisons.

We would highlight that we have already compared with several state-of-the-arts that the reviewer listed, and have highlighted the results in the paper's experiments (Section 4).

Similarly, in real-world data evaluation, there is an absence of comparisons with state-of-the-art STPP models and their mentioned baselines. The authors should consider referring to recent literature for these comparisons to provide a more comprehensive evaluation.

We would highlight that we have compared with state-of-the-art STPP models in the ablation study (Section 4.2.1), which has clearly demonstrated the superior performance of our method against modeling graph point process using models constructed in the Euclidean space.

Although the design of the kernel is motivated by Mercer's theorem explained in Sec. 3.2, and the basis function is represented using neural networks. In practice, we do not need orthogonality for the basis function. The reason can be explained potentially by the fact that as long as the basis function spans the space, it will be a good representation of the kernel; the training of the neural networks is done using stochastic gradient descent, which rarely leads to any nearly linearly dependent basis functions. In our experiment, we have chosen the rank $r$ to be large enough such that the empirical performance is good and validated in Appendix G.

We want to emphasize that our framework can directly incorporate known graph structure; in particular, the observed graph structure information can be leveraged in the construction of the localized graph filter $B_r(v', v)$ in the influence kernel based on the specific formulation of the GNN structures discussed in Appendix CB (e.g., Chebnet, GAT, L3net, GPS Graph Transformer). Indeed, we have demonstrated this using synthetic data experiments in Section 4.1.2; the localized graph filters are created based on the observed graph structures.

We currently do not directly handle ``heterogeneity of nodes in graph,'' we admire this approach in [1] and believe our model can potentially be extended in the future to account for this, for instance, by introducing latent structures to the nodal feature.

We thank the reviewer for the positive comment and acknowledge the value of our proposed model. We have added additional background information on the graph-based kernels to make the paper more self-contained.

Regarding "what types of network interactions can be modeled and what the limitations of such representation are." Currently, we assume the kernel is not "rank-one," with more than one pair of basis functions (over time and graph), the temporal kernel can allow non-stationarity over time, and the graph basis is completely general to enable efficient computational; we assume each basis function is decoupled over "time" and "graph" which is also intuitive and commonly used in the literature; compared with prior models, which usually is "rank-one" our model can be much more expressive while striking similar computational complexity.

One potential limitation is that the current kernel representation still assumes that the influence over events is "additive," which is common in the kernel-based type of methods for point processes. To be more general, one can potentially adopt a non-additive model over events that can be more complex, and we leave it for future work. We have now discussed the limitation in the Conclusion section.