EventFlow: Forecasting Continuous-Time Event Data with Flow Matching

We first would like to thank the reviewer for their valuable feedback, which we have used to improve the latest version of our paper.

I would've liked to see some more comparison against non-autoregressive type models for generation...

We agree that additional non-autoregressive models certainly would be a useful point of comparison. However, to the best of our knowledge, our method EventFlow and Add-and-Thin (which we compare against) are the only two existing non-autoregressive models for unmarked TPPs. Designing and evaluating other classes of models within our framework of predicting the number of events $p(n | H)$ and their times $p_n(t_1, \dots, t_n)$ is an exciting direction for future work.

That being said, we deliberately chose to build on the flow matching framework to design our model. In particular, flow matching has been shown to obtain strong results on complex, high-dimensional domains, e.g. images, while being simpler in terms of the necessary theory and implementation. Thus, our motivation was to obtain a simple yet highly performant model.

Comparatively, MMD wise the metric always measures with respect to the distribution, so with respect to this metric joint distribution models instead of autoregressive ones would have the advantage here.

We would like to emphasize that the MMD metric is only used to evaluate our models when used for unconditional generation, and not forecasting. Moreover, on the forecasting task, where we do not use MMDs, we see a clear gap between the performance of autoregressive and non-autoregressive models.

We also note that our primary focus is on multi-step forecasting tasks, and any evaluation of performance on this task naturally will involve the joint distribution over event times. One-step metrics, like MSE, are unable to capture the multi-step performance we are most interested in.

I would like to see some more discussion on the synthetic datasets with IFTPP...

We thank the reviewer for noting that IFTPP performs better on the unconditional, synthetic task (Table 2). However, we would like to again highlight that our method performs best or second best on 5/6 of these datasets and obtains the highest overall mean rank for the unconditional generation task.

As for why this is the case, our intuition is that this comes down to the complexity of the data -- the synthetic datasets are easier to model than the real-world data. On easier datasets, relatively simple models are sufficient for obtaining strong performance (e.g., IFTPP parametrizes the decoder as a log-normal mixture). The simplicity of the model in this case can be a boon, as simpler models should be more robust and easier to optimize. Conversely, on more difficult datasets, a more complex decoder becomes necessary, and the simplicity of IFTPP becomes a burden.

How does the length affect the corresponding accuracy? Is there an error vs sequence position plot?

We appreciate these interesting suggestions for additional ways of evaluating our method. We have performed an additional experiment, where we now evaluate the forecasting performance of our model across different forecasting horizons. In our preliminary results, we find that our method exhibits an approximately linear degradation in performance as we increase the value of $\Delta T$. In the table shared here, we report the forecasting errors (i.e., as in Table 1), where the columns indicate different values of $\Delta T$.

We plan to perform the same experiment for the baselines to compare the rate at which the error increases, but have not had the time to complete this yet.

While we were unable to investigate how sequence position effects error for this response (due to the limited time allotted), we hope to investigate this further for a camera-ready version of our paper if accepted.

appendix should be in the supplementary material

Please see the FAQ here: https://iclr.cc/Conferences/2025/AuthorGuide -- we opt to have all of our materials in one .pdf file for ease of access.

We thank the reviewer for their feedback, which we have incorporated into the latest version of our paper.

The biggest weakness, in my opinion, results from the use of balanced couplings. Modeling the event count distribution pθ(n∣H) with a deterministic classifier limits the distributional forecast...

We believe there might be a slight misunderstanding here. While the event count distribution $p_\phi(n | \mathcal{H})$ is learned as a classification problem (i.e., by minimizing the cross-entropy loss), at generation time we do not use a deterministic prediction. That is, rather than taking the argmax of $p_\phi(n | \mathcal{H})$, we sample from the corresponding distribution over n. See, for instance, Line 5 in Algorithm 2, and the “Sampling” paragraph in Section 4.4

The paper does not discuss whether a likelihood evaluation is possible... Is computing the likelihood for single sequences possible?

In principle, we believe that one could evaluate likelihoods with our method via the change-of-variables formula. This requires evaluating the likelihood of a prior sequence, which in our setup, is an ordered sequence of i.i.d. random variables. The corresponding likelihood can be found through the corresponding order statistics. However, in our work to date we chose not to use a likelihood-based evaluation, as two of our four baselines (Add-and-Thin and diffusion) are unable to calculate likelihoods. We agree, though, that this could still be a useful evaluation metric. We plan to add likelihood results in an updated version of our paper, but this will require us to add additional baselines which can compute likelihoods, which we have not had time to implement and tune yet given for this response.

Certain hyperparameters are missing. How many neural function evaluations are used to solve the ODE? An ablation would help compare the performance with previous diffusion-based approaches.

We apologize for this oversight in our submission. Our flow time $s$ takes values $s \in [0, 1]$. At training time, $s$ is not discretized, but rather sampled uniformly on $[0, 1]$. This allows us to use an arbitrary choice of ODE solver at sampling time. In practice, we use a Euler solver with 25 uniformly sampled time steps (i.e., 25 NFEs) when sampling. We note that our framework is agnostic to this choice, and other ODE solvers are readily applicable.

We are also grateful for the suggestion to perform an ablation regarding the NFEs. Please also see our global response, we perform an additional ablation where we vary the number of discretization steps in the solver.

A runtime comparison with AddAndThin needs to be included...

We agree that an efficiency comparison would be useful. However, we generally prefer not to report wall-clock times for several reasons. Such times are generally hardware-dependent, can vary based on server load, are challenging to standardize across different codebases, and often depend on implementation details which are independent of the core efficiency of the method. That being said, we generally find that our method has favorable runtimes compared to add-and-thin. For example, to generate 100 batches of 256 sequences on the Yelp-M dataset, add-and-thin required 741 seconds, whereas our method required only 125 seconds. We emphasize, though, that sampling speed efficiency was likely not prioritized in these implementations.

We instead prefer to report the number of function evaluations (NFEs) required to generate a sample. This metric is independent of implementation details and specific architecture choices, and instead highlights the underlying efficiency of the core method.

In particular, our method is able to obtain strong forecasting results using only a single NFE to sample a sequence given $n$. We emphasize that the reference implementation of Add-and-Thin uses 100 NFEs to generate a sequence, diffusion uses 1000 NFEs per generated event, and the autoregressive baselines (IFTPP, NHP) require one NFE per generated event. Please see our global response for additional details.

We thank the reviewer for their detailed review and valuable feedback.

My main concern/weakness is evaluation... I don’t know how the forecasted number of events compared with the ground truth...

We thank the reviewer for suggesting further evaluation of the learned event count distributions.

In the unconditional experiments, we simply sample the number of events $n$ from the training distribution. There is no learning of the event count distribution required here, as this is in a sense the optimal thing to do.

In the conditional experiments, we now need to forecast the number of events conditioned on the history. This step requires learning, where we adopted a simple approach where we learn $p_\phi(n \mid H)$ by minimizing a cross-entropy loss. We agree that further evaluation of our learned event count distribution would be useful. Please see our global response for an evaluation in terms of MARE.

Can the authors clarify at what flow time do the authors use as results reported for your experiments, since s is discretized.

We apologize for this oversight in our submission. Our flow time $s$ takes values $s \in [0, 1]$. At training time, $s$ is not discretized, but rather sampled uniformly on $[0, 1]$. This allows us to use an arbitrary choice of ODE solver at sampling time. In practice, we use a Euler solver with 25 uniformly sampled time steps (i.e., 25 NFEs) when sampling. We note that our framework is agnostic to this choice, and other ODE solvers are readily applicable to a given, trained model.

Please also see our global response, we perform an additional ablation where we vary the number of discretization steps in the solver.

The EventFlow model does not guarantee the generated sequences of event times preserves the original order...

Your understanding is correct -- we do not explicitly enforce that the model produces sequences that are increasing order. At evaluation time, we did not post-process the generated sequences to re-order them.

While a small number of generated events are indeed not ordered, this typically only happens when the generated events are close together in time. Since our main tool for evaluation is the distance in Equation (9), which is in spirit an MAE, this occasional mis-ordering does not have a significant effect on our results. To sanity check this, we re-ran our forecasting evaluation (i.e., corresponding to Table 1) after ordering the outputs of our model. This only changed our results in Table 1 in the second decimal place. To put this into context, this change is smaller than the precision we report in the paper, and significantly smaller than the standard deviation in performance across random seeds.

The author claims the approach is likelihood-free...

We would like to clarify that our method is “likelihood-free” in the sense that training is not based on a maximum likelihood procedure.

In principle, we believe that one could evaluate likelihoods with our method via the change-of-variables formula. This requires evaluating the likelihood of a prior sequence, which in our setup, is an ordered sequence of i.i.d. random variables. The corresponding likelihood can be found through the corresponding order statistics. However, in our work to date we chose not to use a likelihood-based evaluation, as two of our four baselines (Add-and-Thin and diffusion) are unable to calculate likelihoods. We agree, though, that this could still be a useful evaluation metric. We plan to add likelihood results in an updated version of our paper, but this will require us to add additional baselines which can compute likelihoods, which we have not had time to implement and tune yet given for this response.

Please see our discussion with Review u733 for further details regarding likelihood calculations.

[1] Intermittent Demand Forecasting with Deep Renewal Processes, Turkmen et al., 2019 [2] Analysis of Neuronal Spike Trains, Deconstructed, Aljadeff et al., 2016

We thank the reviewer for their detailed comments and valuable suggestions for improving our submission, which we have used to update the latest version of our work.

It is not clear what’s the purpose of introducing balanced coupling of TPPs in Section 4.1...

We agree that the role of the balanced couplings was not made sufficiently clear in our submission. We have added additional clarification in the latest version of our submission in Section 4.2 and Section 4.3

While the interpolation between any two fixed, given sequences does not rely on this coupling (Equations 3-4), the corresponding marginal path of sequence distributions does crucially rely on choosing a coupling, as well as the corresponding marginal vector field (Equation 6). In other words, choosing a coupling is a necessary step in constructing our model.

In Section 4.1, we single out a particular class of couplings -- the balanced couplings -- which allow us to construct a model that does not change the number of events. That is, if we had used an unbalanced coupling, our model would have to somehow incorporate a variable number of events. We explicitly seek to avoid this complexity. Proposition 1 tells us that the only way to obtain such a balanced coupling is if the reference measure $\mu_0$ has the same event count distribution as the data measure $\mu_1$.

I would suggest the authors to consider either clearly separating their original contributions from the methods in existing works or more clearly stating their contributions...

We thank the reviewer for this suggestion regarding improving the presentation of our approach. We have added additional details in Sections 4.3 and 4.4 to better clarify the relationship with existing works.

Note that, while the interpolant construction in Section 4.3 builds on standard flow matching ideas, this is only possible due to our use of a balanced coupling, which allows us to work with a fixed number of events when constructing our interpolants.

The work constrains the type of TPP it models to unmarked temporal point process without event category labels. The practical values of the work is limited. Can the approach be extended to model marked TPP with event category labels in trivial ways...

While our work indeed focuses on unmarked TPPs, we respectfully disagree that the practical value of our work is limited. We explicitly chose to focus on the unmarked setting for several reasons. First, there are many practical applications which do not involve marks, e.g. demand forecasting in online services [1] or modeling neural spike trains [2], as well as the datasets we consider in the submission.

Second, as we discuss in Section 1, our main contribution regards modeling the event times themselves, for which we see significant gains in performance. While we believe that incorporating marks into our model is straightforward, this aspect would not serve as an effective way of evaluating our primary contribution. For instance, as the reviewer suggests, standard techniques would allow us to learn a mark decoder on top of our history encodings, in essence learning a conditional distribution over marks given the event times.

It is arguable that MSE/RMSE for next event prediction is one of the most important evaluation results for TPP models...

We emphasize that our primary motivation is multi-step forecasting problems. Evaluating MSEs for next event prediction is somewhat at odds with this goal, since in the multi-step problem we must predict both the number and times of the events. We note that the cited papers which use MSEs to evaluate all focus on the single-step setting, and hence the appropriateness of MSE as a metric.

However, we provide in Table 7 (Appendix C) an evaluation of one-step-ahead MSEs. We follow the setup of our forecasting experiment, except we now report the corresponding one-step-ahead MSEs. Generally, all of the methods show similar results on this metric, despite there being clear differences between methods on the multi-step task. We believe this serves to further highlight the necessity of moving beyond single-step prediction tasks.

Do the models for conditional and unconditional generation share parameters?

In our experiments, the conditional and unconditional models do not share parameters. However, this is an interesting idea, and sharing parameters across these tasks is relatively straightforward if one so desires. One approach is to use a fixed (or learnable) embedding corresponding to “no history” and training our EventFlow model on a mixture of conditional and unconditional sequences.