Variational Neuro-Symbolic Generative Temporal Point Process

The is one critical weakness in this paper that ultimately led to my “poor” soundness score and overall rating. The proposed method is repeatedly claim superior predictive and generative performance with respect to modeling event sequences when compared to, among others, more traditional neural TPP models; however, as far as I can tell all of these results are invalid comparisons. This is because of how predictions and generations are being made with the proposed approach. This process is only mentioned in passing once, without any alternative mentioned, on line 358: “In the generation phase, after the model being well-trained, we start from an initial state and sample $z$ from $\hat{z}^{(H)}$…” For both prediction and generation tasks this implies that there is a data leakage as $\hat{z}^{(H)}$ is the result of encoding the very sequence being compared against. When performing a sequential VAE, there are generally two high-level approaches to take:

1. Encode the entire sequence, $z = e(x_1, \dots, x_n)$, and then decode the entire sequence $\hat{x}_1, \dots, \hat{x}_n = d(z)$ (could be done auto regressively, or
2. Encode partial histories of the sequence, $z_i = e(x_1, \dots, x_i)$ for $I=1,…,n$, and then partially decode continuations of the sequence, $\hat{x}_i = d(z_i, x_1, \dots, x_{i-1})$.

This work clearly falls under the first scenario, and because of this during prediction to not ‘cheat’ one must sample $z$ from a prior rather than the approximate posterior $q(z | x)$. The take-away from this is to either convert the setup into something more resembling (2), which can iteratively adapt the hidden state as new information comes in, or remain as (1) but put much more focus on the interpretation of the posterior $q(z | x)$. From what I can tell, this is the one valid thing the paper did do with regards to empirical results as there is no data leakage possible here. That being said, there is not enough investigation into this part of the model (as far as I can tell, a small paragraph in Sec 5.4 and a single Table 11) to justify overlooking the prediction and generation issues.

My main concern is the updating of latent variable z for predicates or concepts. The authors just mechanistically describe how they update and leverage forward chain, however I am not sure how the updates are grounded. After applying to H iteration, does z reach some equilibrium? It would be helpful for the authors to provide more details on the theoretical or empirical justification for their update process.

A minor is the authors should briefly discuss how this work differs from previous yang et al, from line 113-114.

• The method bears much similarity to Neuro-Symbolic Temporal Point Processes (Yang 2024), especially consider both use predicate embeddings, represent rules as vector embeddings, and use similarity measures for rule matching. The main difference is the additional binary representation by VAE and Yang et al. uses greedy optimization (per rule) instead of joint optimization of all rules. However, the comparison with Yang et al. is negligible.
• Using VAE for TPP is not new (we have too many VAE-based TPPs); the motivation for VAE here is unclear. It seems like your reasoning process only needs a latent variable (z in your case), so why do we need an additional sampling step? We should be able to directly send encoded x to the reasoning process. The previous work does not need a VAE. The TPP is already stochastic, so you are essentially introducing extra stochasticity without obvious reasons.
• The interpetability here is more or less a "sanity check". Seeing concept satisfaction increases over time is good but cannot really help user make any judgments — we only know that the model is making good use of the concept. When using neural sympolic model, common motivations are checking more straightforward interactions between rules and data, but in your paper it seems such interaction is more much complicated than that of Yang et. al.

Descriptions of the concepts and how they are extracted are underspecified. The authors say that some are pre-specified (line 236) and others can be learned, and they say that they "construct ground truth governing logic rules either by experts or LLMs, with details illustrated in Appendix B". But Appendix B contains only a limited list of concepts, and no details on how pre-specified concepts are chosen or extracted is provided.

Other methodological details are similarly unclear. For example, I could not find details about the size of the latent representation space used in the experiments. It is hard for me to believe that the very limited number of concepts and rules described in the Appendix would be sufficient to achieve good reconstruction performance, so I suspect that the latent representations are high-dimensional. However, if this is the case, it undercuts the interpretability of the method, which is a key advantage noted by the authors.

The concepts and rules presented in the Appendix don't make much sense, and it is not clear to me how they were defined or whether the lists presented are comprehensive. Is the Appendix presenting only a subset of concepts that were cohesive and easy to interpret? And if so, is the method truly interpretable?

The results suggest that the proposed method consistently yields better predictions and more realistic data while still being more interpretable compared to a wide range of established methods. This seems too good to be true, and I don't understand why it would be the case. I understand and agree with the benefits to interpretability, but the authors provide no explanation as to why the approach ought to improve performance compared to alternatives, which makes me suspect that results may have been cherry-picked or other methods may not have been tuned adequately.