COTIC: Embracing Non-uniformity in Event Sequence Data via Multilayer Continuous Convolution

Quality: 1. The authors can motivate the use of 1D-CNN more. The authors mentioned transformers have N^2 memory and computational bottleneck. What about 1D-CNN? It would be nice to see any theoretical statement of the complexity (only empirical experiments are demonstrated.) 2. the authors did not include synthetical experiments. It would be nice to include experiments on standard Hawkes, and proximal graphical event models ( Bhattacharjya et al. proximal graphical event models) two representative models for modeling real world event sequences in a controlled manner. 3. Empirical experiments and ranks. I am not sure about label prediction. COTIC does not seem to predict well. Could the author provide some insight? Maybe due to the hyperparameter α and β.

Clarity: some parts are not clear to me. “The intensity has to be non-negative and should lay in the space of L1 functions. “ Should it be PDF? “expected return time ∆ti,k+1 correspondingly via the following formulas:” No formular associated with return time ∆ti,k+1. Did the authors calculated from the PDF? How do the author make sure κ(t) = 0 for t <0 the neural network?

1. Lack of Novel Technical Contribution: The approach of treating irregular time points with continuous convolutional application echoes methodologies from prior works, like [1]. The proposed method is only around one page.

2. Compared to existing models like Wavenet and the Neural Hawkes model, the improvements, especially in predicting event types, appear to be marginal.

3. The paper's unclear and potentially confusing mathematical notation, like `$L_1$ functions', can hinder comprehension of the model.

4. The core theoretical innovation of the paper, the convolution theory under non-uniform sequences, is inadequately explained. A more robust formulation of the theoretical background can help better understand and potential application of the model. (Section 3.3)

[1] CKCONV: CONTINUOUS KERNEL CONVOLUTION FOR SEQUENTIAL DATA

First of all, the proposed model has several technical defects in terms of modeling events from temporal point processes:

1. An essential nature of point processes is the stochasticity of future event generation, which means that even with a given history, the occurrences of future events are still stochastic. However, despite the good predictive performance of the proposed model, it only provides deterministic predictions of the next event given history (or to say, the embedding), which leads to a significant model loss when it comes to modeling asynchronous events.
2. The conditional intensity function $\lambda$ of the point process enables such stochastic modeling of future events. However, even though the paper proposes a way to model the $\lambda$, there exists a significant problem. According to equation 7, the evaluation of $\lambda$ at any event time $t_k$ will take the $k$-th event into account. However, according to the nature of point processes [1], the $k$-th event is excluded when evaluating $\lambda(t_k)$. Such a self-inclusion has the severe problem of causality. Also, learning $\lambda$ with self-inclusion based on negative log-likelihood will lead to singular results (extremely large values at event times and small values at non-event times). I expect the authors to have some simulations to prove that their conditional intensity function can accurately recover the ground truth (like the $\lambda$ in the classic Hawkes process with exponentially decaying kernel).

Other weaknesses also exist:

3. No simulation or experiments on synthetic datasets, which makes me suspect whether the model can really recover the ground truth.
4. No results of the comparison of negative log-likelihood on testing sets, which is a classic metric in evaluating the goodness of point process modeling.
5. The notation on page 4 is confusing. If I understand right, $j$ should be the index of events which is an integer, but $T_i$ is the upper limit of the time horizon which is a real number. How $j$ takes the value of $T_i$? And $k > j, t_{ik} ≥ t_{jk}$ is also confusing.

[1] Alan G Hawkes. Spectra of some self-exciting and mutually exciting point processes. Biometrika, 58(1):83–90, 1971.

1. The paper is hard to read.

   • In Section 3.1, this part seems to be a typo: "for $k > j$ $t_{ik} ≥ t_{jk}$"
   • In Equation (1) and (2): f() is a function on t and m, but S() is defined on t, Could I ask what is the relationship between f() and S()?
   • In Equation (3): It is unclear why the integral of $\lambda_m(t)$ is 1 since it does not seem to be a PDF. Also, could you please inform the reviewer where this information (integral of $\lambda_m(t)$ is 1) is used in the later derivation?
   • How Equation (4) is derived?
   • The reviewer does not follow Section 3.5. It seems that the loss function is independent from the previous content (Sec 3.1 - Sec 3.4).

2. The core contribution -- continuous convolution -- should be emphasized more and explained clearer. What is the kernel $\kappa()$, how is this continuous convolution related to the well-known CNN? Does the proposed model scalable?

3. More experimental details should be specified, such as the MIMIC data has 65 event types, only 285 sequences with average length at 8.6. Then, how is the deep learning model trained? It seems the dataset is too small to train a deep learning model. Similar question for IPTV dataset.