Neural MJD: Neural Non-Stationary Merton Jump Diffusion for Time Series Prediction

We appreciate the reviewer’s thorough evaluation. We believe that the concerns stem from some technical misunderstandings of our approach, and we value the opportunity to address each point below.

Model discussion

At the beginning of Section 4.1, when the non-stationary MJD is introduced, there is no structural assumption on the five model parameters, so they can literally be any deterministic functions of $t$ (integrability is mentioned later to make the solution well-defined and continuity is provided by the neural network expression). With this nearly arbitrary flexibility, ''fitting data to non-stationary MJD'' does not seem to be a well-defined task. It is like ''fitting a sequence of arrivals to a non-stationary Poisson process with arbitrary $\lambda_t$'', which leads to sheer overfitting.

Well-defined task concern

We respectfully disagree with the statement that fitting a non-stationary MJD is not a well-defined task.

Previous models, such as Neural Jump Stochastic Differential Equations [1] and Neural Jump-Diffusion Temporal Point Processes (NJDTPP) [2], adopt highly similar processes to learn SDEs from data using neural networks. We find the claim of the task not being "well-defined" to be a vague assessment that contradicts established findings in the research community.

Specifically, in our implementation, we enforce that all learned parameters satisfy the classical MJD requirements. For example, jump intensities and variances are constrained to remain non-negative. This effectively regularizes the learning of SDE parameters.

Overfitting concern

We believe the reviewer’s comparison to "fitting a sequence of arrivals to a non-stationary Poisson process with arbitrary $\lambda_t$" may stem from a misunderstanding. Rather than fitting each series independently (e.g., as done in ARIMA), we train a single neural network using all sequences in the training data. This approach encourages the model to learn shared non-stationary dynamics, preventing it from merely memorizing individual trajectories.

In practice, we use non-overlapping train/validation/test splits and select model checkpoints based solely on validation performance to avoid overfitting. The experimental results on the unseen testing set clearly show no sign of what is referred to as "sheer overfitting."

The expression (8) $\mu_t,\sigma_t,\lambda_t,\gamma_t,\nu_t=f_\theta(S_0,S_{-1},...,S_{-T_p},C,t)$ may also be problematic. The deterministic future parameters $\mu_t,\sigma_t,\lambda_t,\gamma_t,\nu_t$ is expressed as a deterministic function ($f_\theta$) of past sample path ($S_0,S_{-1},...,S_{-T_p}$), which is a sequence of random variables. This violates the internal logic of the SDE (unless those parameters are some adaptive processes, not mentioned in the paper), especially when $\mu_t,\sigma_t,\lambda_t,\gamma_t,\nu_t$ are generated from $S_0,S_{-1},...,S_{-T_p}$ in a sliding-window manner as in Section 5.1. Defining instead predicting $\mu_t,\sigma_t,\lambda_t,\gamma_t,\nu_t$ by (8) (in a sliding-window manner) makes more sense to me, but that will lead to a much more complicated SDE (7) + (8) than non-stationary MJD with deterministic $\mu_t,\sigma_t,\lambda_t,\gamma_t,\nu_t$, so all equations after (8) no longer holds.

SDE logic concern

We respectfully disagree that our method violates the internal logic of the SDE.

To avoid confusion, we would like to emphasize that we do not fit jump-diffusion parameters by directly calibrating them on historical data. Instead, past observations $S_{-1},...,S_{-T_p}$ serve only as inputs to a shared neural network. In other words, they are not part of the SDE state variables to be modeled. They can be incorporated into the context information, denoted by $\mathcal{C}$, in our setup.

To be specific, in our model, the SDE process at time $t$ depends only on information available up to time $t$, with no "peeking" into the future. The model has no exposure to future data before making predictions, which is standard practice in deep learning. All the SDE parameters are predicted based on past observations using neural networks once and for all.

We noted some ambiguity in the comment. Our model was described as "sliding-window," yet the reviewer also suggested that "sliding-window makes more sense." We interpret this as a possible preference for an autoregressive approach (please kindly clarify if we have misunderstood the review).

That said, we would like to clarify that our model does not operate in a sliding-window manner. Instead, it adopts a feedforward design that (1) predicts the SDE parameters for the target horizon and then (2) performs numerical simulation to generate actual forecasts. This architecture allows for a single forward pass to predict SDE parameters across all future time steps under consideration, significantly improving inference efficiency by using neural network only once.

Open source

''Open access to data and code, Answer: Yes, Justification: We will release our code once the paper is accepted.'' Why?

We promise to release our full codebase and data upon acceptance since we would like researchers to be able to reproduce our work. We will release our full codebase and data upon acceptance, as we expect to include additional experiments and corresponding code during the rebuttal phase.

Concluding remarks

We thank the reviewer again for their valuable feedback. We hope that our rebuttal addresses their questions and concerns, and we kindly ask the reviewer to consider a fresher evaluation of our paper if the reviewer is satisfied with our responses. We are also more than happy to answer any further questions that arise.

Reference

[1] Jia, Junteng, and Austin R. Benson. "Neural jump stochastic differential equations." Advances in Neural Information Processing Systems 32 (2019).

[2] Zhang, Shuai, et al. "Neural jump-diffusion temporal point processes." Forty-first International Conference on Machine Learning. 2024.

We thank the reviewer for engaging in the discussion.

Amortized vs. Standard Statistical Inference

First, we would like to clarify that our goal is not to merely "fit to the standard paradigm of statistical inference." Instead, we adopt the amortized inference principle by training a shared neural network that maps context features to SDE parameters. This neural network is trained across a large, diverse dataset capturing a wide range of contextual variability. At test time, the pre-trained network takes in unseen context features and outputs parameter estimates in a single forward pass, without any additional optimization.

Conditional vs. Unconditional Modelling

Second, our model is trained using a conditional MLE objective. It conditions on a past window of time-series observations, and optionally on static features (e.g., store attributes in business revenue prediction). This stands in contrast to the reviewer's toy scenario, which assumes no conditional input. In real-world datasets such as stock prices or store revenues, each sample typically has non-overlapping context features; identical context configurations are not to be found in our actual datasets, excluding one-to-many ambiguity in implementation.

Overfitting

Regarding the reviewer's overfitting concern in the non-stationary Poisson process example, we respectfully disagree with the conclusion. In the second case, involving a model family with all-positive $\Lambda$ functions, the estimated $\lambda_\star$ achieves the lowest training error by capturing four bumps. However, low training error (i.e., good fit or perfect fit) does not, on its own, imply overfitting (i.e., poor generalization to unseen data). In the toy example provided, there is no quantitative evaluation or empirical evidence of generalization performance on held-out data. While we acknowledge that more flexible model families can achieve lower training error, this does not necessarily lead to poor generalization—it must be assessed through explicit experimental validation.

On the contrary, prior research shows that perfect fitting does not necessarily hinder generalization for neural models. In [1], the authors demonstrated the phenomenon of benign overfitting, showing that least-squares interpolators can generalize well despite zero training error in certain high-dimensional linear models. In [2], it was shown that large neural networks can memorize random labels while still generalizing effectively on real data. Moreover, we observe no signs of overfitting in our experiments, as our model consistently achieves competitive quantitative performance compared to various baselines.

Overall, we thank the reviewer for engaging in the discussion and hope our rebuttal has addressed the concerns raised. We remain open to further dialogue.

Reference:
[1] Bartlett, Peter L., et al. "Benign overfitting in linear regression." Proceedings of the National Academy of Sciences. 2020.

[2] Zhang, Chiyuan, et al. "Understanding deep learning requires rethinking generalization." International Conference on Learning Representations. 2017.

We sincerely thank the reviewer for their time and effort in evaluating our work. We greatly appreciate their recognition of our model formulation and the application of MJD SDE modeling to time-series analysis. Below, we address the key points raised in the review.

Confidence interval

I would like to see an evaluation where, if the best models are performing similar (statistically significantly), they are both in bold.

You sample 10 times to express mean metrics. Can you please also provide confidence intervals on these metrics? E.g. using t-test statistics as is done in Bartosh SDE Matching (2025) and earlier works on latent SDEs.

Thank you for your valuable suggestions. We provide additional experimental results on the SafeGraph & Advan business analytics dataset, where we compute the confidence interval by calculating the standard deviation across multiple runs. The preliminary experiments are as follows:

Due to the time constraints of the rebuttal period, we will compute the t-test statistics as suggested in future work and include these metrics for all stochastic methods in additional experiments.

Synthetic dataset

The first experiment is on synthetic data, i.e., you know the true parameters of the MJD process that generates the data. Can you please show how the estimated parameters using your method compare to the true parameters?

Thank you for your feedback. We show the estimation parameter differences for the $\mu$ and $\sigma$ parameters using our Neural BS and full Neural MJD models. The results are averaged over time steps to obtain aggregated MAE results.

As we can see, the Neural MJD model performs slightly better, likely due to its ability to model jumps during the learning process, whereas the Neural BS model only captures the SDE without jumps. For the other jump-related parameters, we don’t observe very close estimations from the Neural MJD. We hypothesize that different combinations of jump parameters may yield similar outcomes in the likelihood loss training. We plan to devote more effort to exploring this aspect further in order to enhance model interpretability. Thank you again for your insightful comment.

Restart mechanism

You restart the EM solver at each observation. Do you think this has a negative effect on learning long-term dependencies? I mean, a change of the parameters at some time t will not have an effect on the dynamics after the next S_\tau is crossed?

We would like to respectfully point out that the restart mechanism is based on the conditional mean, which is given by:

$$\mathbb{E}\left[S_{t} \vert \mathcal{C} \right] = S_0 \exp(\sum_{j=1}^{\rho_t-1} \mu_{j} + (t-\rho_t+1) \mu_{\rho_t}).$$

As such, the SDE parameters $\mu_t$ still implicitly capture the long-term dependencies learned by the neural network.

We agree that changes in other parameters have a lesser impact on the SDE dynamics once $S_\tau$ is crossed. This observation aligns with our design motivation, which aims to reduce simulation variance in the long term to achieve improved empirical performance.

Regularization loss

I'm a bit surprised you needed to add the 'regularization' term. It resembles a Gaussian likelihood on the measurements (L2 term). Can you provide me more intuition on why (or why not) your likelihood does not already contains a similar effect?

Thanks for your comment. The extra regularization loss is added to address the practical issues associated with the likelihood-based learning objective. The intuition behind this is that a low likelihood loss does not necessarily guarantee that the conditional mean (as represented by the regularization loss) is well learned. Due to the jump components of the SDEs, we conjecture that there may be local minima that result in a relatively low likelihood loss, but where the conditional mean can still vary significantly. Since our restart solver also uses the conditional mean, we find that adding this term generally enhances performance by constraining the optimization space when using likelihood-based loss.

Related work

The related work on jump processes seems to focus on the financial application domain. But inference of jump processes has been investigated in machine learning, how does your work relate to these 'older' works?

We thank the reviewer for highlighting these foundational works on jump-process inference. It is an insightful suggestion to include a comparison with classical inference methods for jump processes.

While these methods share the goal of estimating non-stationary parameters from discrete observations, our approach differs in three key respects:

1. Inputs vs. Calibration: We do not fit SDE parameters directly to past data using Maximum Likelihood Estimation (MLE). Instead, we treat historical observations and additional information as inputs to the neural network. The likelihood serves as a training loss to optimize forecast accuracy, rather than as a per-series estimator of past states.

2. Classical Methods: Classical Expectation-Maximization (EM) or Markov Chain Monte Carlo (MCMC)-based methods estimate parameters for each individual sequence. In contrast, our single network is trained once over all series, capturing shared dynamics and generalizing across assets or nodes.

3. Closed-Form Requirement: Classical methods do not require a closed-form transition density, while our method does. This limitation provides a promising direction for future work, where the closed-form constraint could potentially be relaxed.

These "older" works provide a strong statistical foundation and could inspire hybrid methods. We will incorporate a discussion of their contributions and distinctions in the Related Work section of the revision.

Additionally, we find that these works offer valuable insights into the challenges and non-uniqueness of estimating non-stationary event rates from discrete observations (which relates to the previous question).

Writing

line 178, this means you always discretize with time-step equal to the time unit?

In practice, we hold the parameters fixed during the time unit (1 in this case), but during this time unit, you can choose any smaller $\delta$ to reduce discretization error.

We thank the reviewer for pointing out the typos and formatting issues. We will address these in the revision.

Concluding Remarks

We sincerely thank the reviewer once again for their valuable feedback. We hope our responses have satisfactorily addressed their questions and concerns. We remain fully available and happy to answer any further questions that may arise.

We would like to express our sincere gratitude to the reviewer for their time and effort in evaluating our work. We are grateful for their acknowledgment of the novelty in our model formulation and the application of MJD SDE modeling to time-series analysis. Below, we respond to the key points raised in the review.

Additional experiments

The S & P experiment is run on a limited set of data (Jan 2016 - Apr 2017), which had limited volatility (evidenced by the VIX index during this time), it is unclear whether this method still outperforms given the increased volatility of the past 5 years.

Thank you for your valuable feedback. We have extended the S&P 500 dataset to cover the period from January 2014 to December 2017, using the data from 2017 for performance evaluation. The preliminary experimental results are as follows:

In the extended dataset, our method still outperforms the baselines. In future work, we plan to extend our experiments to cover the past five years comprehensively, alongside additional baselines. We will also analyze the empirical effects of long-term forecasting and data volatility on model performance.

Discussion on related work

Additional discussion on the relationship between this work and other works relating neural learning in non-stationary time series would be appreciated.

We thank the reviewer for the helpful suggestion to clarify how our work relates to prior research on non-stationary time series. We will include this clarification in our revision to prevent any ambiguity.

In previous studies, "non-stationarity" typically refers to distributional shifts in the data over time. These works focus on mitigating such shifts using techniques like input-level normalization (e.g., DAIN [1], ST-norm [2], RevIN [3]), domain adaptation (e.g., DDG-DA [4]), or de-stationary attention mechanisms (e.g., Non-stationary Transformers [5]).

In contrast, our notion of non-stationarity centers on modeling a Merton jump diffusion (MJD) process with parameters that evolve over time. While our model is not explicitly designed to handle distributional shifts, it predicts time-varying MJD parameters through a closed-form solution, which naturally leads to a changing predictive distribution as parameters evolve.

Model implications

What are the implications of this work for modeling option prices and more generally volatility?

Thank you for your feedback. We would like to highlight the following advantages of our approach:

1. Reduced calibration effort: Classical option-pricing methods require significant effort to fit a time-inhomogeneous Merton jump-diffusion model to market or historical data. In contrast, Neural MJD treats past returns as conditioning inputs rather than direct calibration targets and directly outputs the drift, diffusion, and jump-intensity parameters for the future time horizon. This approach eliminates the need of time-consuming calibration due to fast neural network inference.

2. Handling exogenous information: Most classical modeling methods either ignore exogenous drivers or can only handle numerical features. Our method, however, is capable of incorporating diverse types of data as inputs or conditioning variables, such as static features (e.g., parking lot sizes) in the SafeGraph and Advan datasets used for business revenue prediction.

3. Potential extension to other SDE models: A similar neural parameterization technique can be extended to other SDE dynamics for more domain-specific applications, allowing for better capture of volatility. The training and inference techniques introduced in the paper would be beneficial for future model development.

Theorem proof

In the proof of Theorem 4.1, what are the precise assumptions necessary for this theorem to hold?

Theorem 4.1 (the truncation-error bound for the non-stationary MJD) is proven under the following assumptions:

1. We choose $\delta$ small enough that $t$ and $t+\delta$ are under the same set of parameters. We have this assumption simply to ease the proof of the theorem.
2. Other standard Poisson/MJD assumptions: a. The jump sizes in Poisson process are i.i.d. b. Conditional on $\Delta N$, $\ln S_{t+\delta}$ is Gaussian.

We will clarify the assumptions used in the proof of the theorem, and we thank the reviewer for bringing this up.

Discretization interval

Does the method obtain improved results when the discretization is performed on smaller intervals?

Theoretically, using a finer discretization reduces both the Euler integration error and the jump-truncation bias per interval. Empirically, we find that using an interval that is small enough (such as 0.1 or 0.2) leads to good performance. We conduct the comparison on the S&P 500 dataset as follows. Please note that the interval is normalized relative to the time difference between two times of interest (which is 1 in our notation).

Concluding Remarks

We sincerely thank the reviewer once again for their valuable feedback. We hope our responses have satisfactorily addressed their questions and concerns. We remain fully available and happy to answer any further questions that may arise.

Reference

[1] Passalis, Nikolaos, et al. "Deep adaptive input normalization for time series forecasting." IEEE transactions on neural networks and learning systems 31.9 (2019): 3760-3765.

[2] Deng, Jinliang, et al. "St-norm: Spatial and temporal normalization for multi-variate time series forecasting." Proceedings of the 27th ACM SIGKDD conference on knowledge discovery & data mining. 2021.

[3] Kim, Taesung, et al. "Reversible instance normalization for accurate time-series forecasting against distribution shift." International conference on learning representations. 2021.

[4] Li, Wendi, et al. "Ddg-da: Data distribution generation for predictable concept drift adaptation." Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 36. No. 4. 2022.

[5] Liu, Yong, et al. "Non-stationary transformers: Exploring the stationarity in time series forecasting." Advances in neural information processing systems 35 (2022): 9881-9893.

We sincerely thank the reviewer for their time and effort in evaluating our work. We appreciate their recognition of the novelty of our model formulation and the use of SDE modeling for time-series analysis. Below, we address the main points raised in the review.

Applicable domain

benchmarked on some non-financial timeseries

To further address this concern, beyond the economic-financial time-series data used in the paper, we conducted an additional experiment predicting daily visit volumes to business points of interest (POIs) using the SafeGraph and Advan datasets. Our method continues to outperform the baseline models in this setting as well.

As future work, we aim to apply our method to more diverse settings and datasets, such as healthcare datasets as suggested by the reviewer.

more specialized family of differential equations

We respectfully argue that our approach, inspired by Merton jump diffusion SDEs formulations, provides a broader and more flexible framework than simpler models like the Black-Scholes SDE. This allows our model to better capture jump behavior, which standard neuralized SDEs may struggle with. While fully neural formulations (e.g., NeuralCDE, LatentSDE) can theoretically represent a wider range of SDE processes, they do not outperform our approach as seen in the experiments. Empirical results demonstrate that the use of jump diffusion SDEs in our model design leads to improved performance.

Baseline fairness

I would be much more interested in comparisons with state-of-the-art forecasting methods

We conducted further experiments on the S&P 500 dataset and obtained preliminary results for TCN and N-BEATS:

We plan to include additional experiments comparing with more baselines in the revised version.

I think the baselines are sort of set up to fail.

We would like to clarify that our experiments were designed for a fair comparison. Some baselines do not explicitly model jumps, motivating our Neural MJD, and the performance comparison supports the proposed model. Except for the synthetic data, we do not control jump behavior in real-world datasets. Our goal is to ensure a fair, practical comparison on real data without any jump labels and to highlight the methodological value of modeling potential jumps.

What is the “MLP” baseline.

The MLP model consists of a stack of linear layers and serves as a simple baseline for time-series forecasting.

Are the time series used univariate or multivariate?

We use univariate time-series datasets, predicting a scalar at each time step.

Generalization to non-stationary data

I do not agree with some of the parlance like “inability to explicitly model underlying stochastic processes often limit their generalization to non-stationary data”.

We thank the reviewer for their valuable feedback. Our intention was to emphasize that explicitly modeling jump diffusion SDEs offers a useful inductive bias for capturing sudden changes and non-stationary behaviors that may be more challenging to represent with standard neural SDEs. We believe this approach complements the neural differential equation framework by providing additional flexibility. We appreciate the suggestion and will revise the manuscript to ensure accuracy.

Neural ODE vs. ours

how the model produces inferences for any t with a single evaluation?

why do you need to do the integration steps at all?

We would like to clarify that our model is not a neural ODE and does not require step-by-step integration of a neural vector field. Instead, it predicts all SDE parameters—drift, diffusion, and jump intensity—in a single forward pass. The SDE trajectory $S_t$ is then simulated using these predicted coefficients via a numerical method, without further neural network evaluations.

A neural ODE typically learns a vector field: $\frac{dS(t)}{dt} = f(S(t), t; \theta),$ and computes $S(T_f) = S(0) + \int_0^{T_f} f(S(t), t; \theta)\,dt$ using an ODE solver, which repeatedly queries the neural network during integration.

In our method, we do not learn a neural vector field. Instead, we predict the full set of Merton jump-diffusion (MJD) parameters: $[\mu_1, \sigma_1, \lambda_1, \gamma_1, \nu_1, \dots] \in \mathbb{R}^{5 \times T_f}.$ Due to the stochastic nature of SDEs, numerical simulation is required to generate actual forecasts. We compute each next state (informally) as: $S_{t+\Delta t} = S_t + \mu_t\,\Delta t + \sigma_t\,\Delta W + \text{jump}(\lambda_t, \gamma_t, \nu_t),$ which involves stochastic integration—but no further neural network inference, as all parameters are predicted once at the start.

In summary, our model generates predictions for the SDE parameters at time $t$, followed by numerical simulations to obtain SDE trajectories for forecasting outcomes.

Why not just evaluate the exact target value and their variances at the times of interest?

We would like to emphasize that we do not forecast outcomes in a single step. Instead, we sequentially sample each increment across the entire SDE simulation trajectory to obtain stochastic forecasts. Relying solely on the exact expectation and variance would oversimplify the model, ignoring the jump dynamics. While our formulation allows for the computation of mean and variance, these values may deviate from the true behavior, which includes jumps.

get a smooth estimate

We thank the reviewer for raising this point. In Tables 1–3, the "Mean" metrics represent averages over 10 runs of the SDE simulation, serving as a simple proxy for smoothing. We agree that more advanced approaches, such as GP fitting, are promising and plan to explore them in future work.

Writing clarity

Thank you for your helpful comments. We will clarify the derivations and notation in the main paper and present our method more clearly in the revision. We followed the classical literature for the Poisson random measure $N(\mathrm{d}t, \mathrm{d}y)$ and will also include explanations highlighting its meaning and interpretation in practical applications to enhance understanding.

Concluding remarks

We thank the reviewer again for their valuable feedback. We hope that our rebuttal addresses their questions and concerns, and we kindly ask the reviewer to consider a fresher evaluation of our paper if the reviewer is satisfied with our responses. We are also more than happy to answer any further questions that arise.