xLSTM-Mixer: Multivariate Time Series Forecasting by Mixing via Scalar Memories

Dear Reviewer cGWV,

Thank you for your strongly affirmative review and for recognizing the novelty, clarity, and empirical efficacy of xLSTM-Mixer.

We are happy to provide further insights into the initial token embedding:

W1 (necessity): The benefit of learning $\mathbf\eta$ can be observed by comparing ablation #7 to the base xLSTM-Mixer model #1 in Table 2. Model #1 consistently performs at least as well as #7 and, in 13 out of 16 cases, learning the initial token embedding yields strictly better results. While the effect is not dramatic, it offers a consistent benefit: on average, learning the initial token embedding reduces MAE by 0.4% and MSE by 0.7% across all eight settings.

W1 (interpretability): We are pleased to expand on the interpretation of Fig. 5, as outlined in lines 237–240. A row-by-row inspection of the figure reveals that similar patterns are learned per dataset, albeit at different scales. Specifically, these patterns repeat in proportion to the forecast horizon. The fact that essentially the same patterns are learned for the same datasets across different initializations and horizons supports the conclusion that they are data-driven and meaningful to the domain.

We also invite you to review our response to Reviewer 67JL, where we significantly expanded our empirical analysis on the GIFT-eval benchmark, reaching impressive leaderboard placements (Q2), conducted statistical significance tests (Q4), and extended the experiments on variate orderings (Q5).

Thank you again for your overwhelmingly positive review. In light of this, we would greatly appreciate it if you could consider raising your score to reflect your assessment.

Best

The Authors

Dear Reviewer cGWV,

Thank you for your helpful review and positive assessment of the work. Your comments sharpened our analysis of the initial embedding and strengthened the paper. The camera-ready version will reflect these improvements.

Bests

The Authors

Dear Reviewer fZrP,

We thank the reviewer for recognizing the thoughtful design and thorough empirical validation of xLSTM-Mixer.

W1: Thank you for mentioning this interesting perspective. This multi-view prediction is primarily related to multi-task learning (Zhang et al., 2022), which is similar to data augmentation in its goal to regularize training. Mixing operations can be interpreted as a latent data augmentation or alternatively as ensembling with weight-sharing as suggested by Reviewer 67JL. We can imagine these perspectives as a guide for future improvements to xLSTM-Mixer, and we will add them to Sec. 3.3.

W2: Yes, we indeed conducted the very extensive real-world benchmark GIFT-Eval (Aksu et al., 2024) encompassing 97 scenarios. It also includes missing values as you suggested, empirically strengthening our contribution. Furthermore, multiple benchmark datasets are also non-stationary. We also note that the existing benchmark datasets already exhibit a diverse range of substantial levels of noise. This is also shown by the Hurst exponent in Appendix A, Tab. 3. We added this discussion to the limitations section of the manuscript as laid out at the end of this response.

W3: Explainable AI is indeed an important aspect, and often at conflict with model fidelity. On one hand, the multistage approach makes certain interpretations harder, like for any deep learning model. See also the limitations section at the end of this response. On the other hand, it permits, for instance, interpreting the initial token embeddings as shown in ll. 234ff and in Appendix D. Overall, attribution methods, such as Integrated Gradients and SHAP, still function as normal, permitting inspection of the learned patterns, such as shown in ll. 266ff and in Appendix E.

W4: Highlighting the differences of the variants in the ablation by including performance drops relative to the full xLSTM-Mixer baseline (#1) is a very useful suggestion. We will improve Sec. 4.4 by adding the following information.

Ablating components of xLSTM-Mixer each causes both error metrics to increase, entailing that they contribute positively to the overall architecture. Specifically, ommiting xLSTM (only having LSTM, #3) raises the MAE by 6.2% / the MSE by 7.0%, variate recurrence (#5) by 4.3%/4.7%, learning initial token embedding (#7) by 0.4%/0.7%, view mixing (#8) by 0.6%/0.7%, and time mixing (#11) by 2.7%/3.1%.

W5: Thank you for mentioning this. Reviewer V19p raised a similar point. Please refer to the response of Reviewr V19p Q1, where we clarify and give an intuition why state‑tracking via memory mixing in sLSTM is indispensable, whereas mLSTM's design cannot provide this functionality.

Comments on Limitations: We welcome your comments on the limitations of xLSTM-Mixer. We added the following to the paper: (1) A dedicated limitations paragraph in Section 6 and (2) a dedicated section on broader societal impact as encouraged by NeurIPS. We attached them both to this response:

Limitations. While xLSTM‑Mixer achieves strong accuracy with limited compute, certain assumptions need to be met for it to be applicable. Specifically, it assumes that all variables are sampled on a uniform time grid, meaning that irregular or missing timestamps must still be handled in pre‑processing. Furthermore, treating variates as the sequential axis ties runtime and memory directly to the number of channels, which can become a bottleneck for extremely high‑dimensional multivariate time‑series. Moreover, the simultaneous mixing of multiple views blends temporal and cross‑channel information in ways that make detailed attributions difficult and may leave the network sensitive to regime shifts or extremely noisy signals. Addressing these aspects through adaptive variate grouping, continuous‑time embeddings, and lightweight explanation modules are potential paths forward beyond the current work.

Societal Impact. This paper presents work whose goal is to advance the field of Machine Learning. There are many potential societal consequences of our work, none of which we feel must be specifically highlighted here.

Our research advances machine learning by enhancing the capabilities of long-term forecasting in time series models, significantly improving both accuracy and efficiency. By developing xLSTM-Mixer, we introduce a robust framework that can be applied across various industries, including finance, healthcare, energy, and logistics. The improved forecasting accuracy enables better decision-making in critical areas, such as optimizing resource allocation, predicting market trends, and managing risk.

However, we also recognize the potential risks associated with the misuse of these advanced models. Time series forecasting models could be leveraged for malicious purposes, especially when applied at scale. For example, in the financial sector, adversarial agents might manipulate forecasts to create market instability. In political or social contexts, these models could be exploited to predict and influence public opinion or destabilize economies. Additionally, the application of these models in sensitive domains like healthcare and security may lead to unintended consequences if not carefully regulated and ethically deployed.

Therefore, it is essential that the use of xLSTM-Mixer, like all machine learning technologies, is guided by responsible practices and ethical considerations. We encourage stakeholders to adopt rigorous evaluation processes to ensure fairness, transparency, and accountability in its deployment, and to remain vigilant to the broader societal implications of time series forecasting technologies.

Thank you again for your thoughtful review. We hope this review answered all outstanding questions. Specifically, having improved the clarity of the work (theory and writing) and provided impressive new benchmark results on the more practical GIFT-Eval. Furthermore, we added a new limitation section and a more extensive discussion of ablation results. Therefore, we have addressed all key concerns regarding our work. In light of this, we would greatly appreciate it if you could raise your score to reflect this.

Best

The Authors

References:

• Zhang et al. A Survey on Multi-Task Learning. IEEE TKDE, 34(12):5586–5609, 2022.
• Aksu et al. GIFT-Eval: A Benchmark for General Time Series Forecasting Model Evaluation. NeurIPS Workshop on Time Series in the Age of Large Models, 2024.
• Beck et al. xLSTM: Extended Long Short-Term Memory. NeurIPS, 2024.
• Greff et al. LSTM: A Search Space Odyssey. IEEE T-NNLS, 2017.
• Merrill et al. Unlocking State-Tracking in Linear RNNs Through Negative Eigenvalues. NeurIPS Workshop on MML, 2024.
• Alharthi et al. xLSTMTime: Long-Term Time Series Forecasting with xLSTM. MDPI Sensors, 2024.
• Auer et al. TiRex: Zero-Shot Forecasting Across Long and Short Horizons with Enhanced In-Context Learning. arXiv:2505.23719, 2025.

Dear Reviewer V19p,

We are grateful for the recognition of the clear writing, the novel xLSTM-Mixer architecture, and excellent modeling capabilities while remaining efficient.

We gladly clarify the raised comments and questions:

W1: We ground xLSTM‑Mixer in time‑ and variate‑mixing theory. A single shared NLinear forecaster that marches over the variates (Section 3.1, motivated by Section 2.2) imposes a strong channel‑independent inductive bias. After transposing the embedding—as shown by the inverted‑token iTransformer (Liu et al., 2023)—the sLSTM stack (Section 3.2) jointly mixes time and variates while keeping parameters and memory low. We then produce two complementary latent views (original and reversed), process both with the same sLSTM, and fuse them with one linear layer (Section 3.3); this weight‑tied multi‑view design follows the regularisation principles of multi‑task learning described by Zhang & Yang (2022). Analytically, block‑diagonal recurrence keeps each head focused on long‑range dynamics, while a learnable soft‑prompt token speeds adaptation—a mechanism also used in TEMPO (Cao et al., 2023). Together, these choices explain xLSTM‑Mixer’s state‑of‑the‑art long‑horizon accuracy and efficiency; additional theoretical insights appear in our response to Q1.

W2: We welcome the suggestion to improve the manuscript even further by outlining xLSTM-Mixer's limitations. We amended the paper with a new section. Please refer to Reviewer fZrP Response for Comments on Limitations for the explicit paragraph.

W3: We initially only provided an outlook to probabilistic forecasting. We welcome your suggestion, and have since then massively extended this to the novel and large probabilistic GIFT-Eval benchmark (Aksu et al., 2024), as also suggested by Reviewer 67JL. In summary, we are pleased to report that xLSTM-Mixer surpasses classical deep-learning and statistical baselines by a large margin on univariate and multivariate datasets, while matching the strongest pre-trained models. Please refer to Reviewer 67JL's response for the benchmark tables.

We are grateful for this insightful suggestion, which has broadened the scope of our work and strengthened the manuscript. We will include the full results in the camera-ready version and publish them on the public leaderboard.

W4: We agree that this is an intriguing idea for further in-depth analysis. These different views can be seen as ensembling with weight-sharing. Therefore, extending the size of the ensemble is a natural next step. We thus performed additional variate experiments with permutations of the variates beyond one (standard order) and two (standard and reversed). Please refer to Reviewer 67JL's response for the scaling summary tables. Again, using two views works best for all prediction lengths, while further increasing the ensemble size can also be beneficial over the single view. However, this may over-regularize the model for some prediction lengths and higher ensemble counts. We observed this on multiple datasets and show ETTh2 as a representative sample. Thank you again for suggesting this insightful perspective. We will include this in our revised manuscript.

W5: For the main comparison, we ensured that the comparison was against the state-of-the-art in each model group (recurrent, convolutional, etc.). Those are not necessarily the newest models. For instance, PatchTST is still the best model in the "deep-learning" category of the GIFT-eval leaderboard. However, we still include three models from 2025 to reflect the latest developments. Regarding the three concurrent works you suggested: TimeCMA shows worse performance than xLSTM-Mixer in all 48 comparable settings (Liu et al., 2025, Tab. 1). HDT is a model solely for probabilistic forecasting (Shibo et al., 2025), and thus not a natural baseline to compare against. Similarly to TimeCMA, xLSTM-Mixer outperforms P-sLSTM in all 24 comparable settings (Kong et al., 2025, Tab. 2). However, we gladly incorporate these three recent works into our related work, especially Kong et al. (2025), which is closely related. As already discussed in W3, we substantially expanded the empirical findings to the GIFT-eval benchmark, thereby adding new baselines via its leaderboard.

Q1: Thank you for raising this point. Beck et al. (2024) provide an in-depth discussion of why xLSTMs are beneficial over previous recurrent models. Nevertheless, below, we clarify and give an intuition as to why we use sLSTM over mLSTMs. The core reason for that is the explicit state‑tracking via memory mixing in sLSTM, which is lacking from mLSTMs:

• sLSTM keeps the hidden‑to‑hidden (memory mixing) connections of the classic LSTM and augments them with exponential gating and stabilisation (Beck et al. 2024; Greff et al. 2017). $$c_t = f_t c_{t-1} + i_t z_t,\qquad h_t = o_t \, g(c_t)$$ Because $f_t$, $i_t$, and $o_t$ all depend on the previous hidden state $h_{t-1}$, the block can learn to blend past and present information on demand.
• mLSTM stores key–value pairs in a matrix $$C_t = f_t C_{t-1} + i_t v_t k_t^T .$$ But to enable full parallelisation, it's gates depend only on the current input, i.e., all hidden‑to‑hidden paths are removed (Beck et al. 2024, Sec 2.3). Thus, memory mixing is absent.

Why this matters for forecasting: Time‑series forecasting often requires the model to keep track of latent phases such as "heating ↔ cooling" or "upward ↔ downward" movement. The model must retain that phase until evidence of a switch appears and flip its internal indicator when the switch occurs.

• In sLSTM, gate values can be trained so that when a switch indicator is detected $i_t = 0$, $f_t = 1$ (overwrite), otherwise $i_t = 1$, $f_t = 0$ (retain). Because the gates read $h_{t-1}$, the decision depends on the current internal phase as well as the new input, enabling reliable phase tracking.
• In mLSTM, gate values cannot depend on $h_{t-1}$. Instead, the update is a linear function of the input history, i.e., it can integrate evidence but cannot implement a conditional flip. Empirically, models without memory mixing (mLSTM, Mamba, Transformers) fail even simple state‑tracking tasks such as parity (Merrill et al. 2024; Beck et al. 2024).

An illustrative Example: Consider a binary series where $x_t = 1$ signals a phase switch; the desired hidden phase $s_t$ obeys $s_t = s_{t-1} \oplus x_t$, where $\oplus$ denotes the XOR operation.

sLSTM case: Choose weights so that $f_t = \sigma(\alpha(1-x_t))$ and $i_t = \sigma(\alpha x_t)$ with large $\alpha$. Let the output gate turn $c_t$ into two well‑separated values (e.g., −1 vs 1). Then $c_t$ flips sign exactly when $x_t=1$, reproducing $s_t$. Because $f_t$ and $i_t$ both read $h_{t-1}$, more complex phase logic (e.g., multi‑step confirmation) can also be learned.

mLSTM case: Unfolding the recurrence shows that $C_T$ is a weighted sum of outer products of current inputs. In other words, the weights are fixed functions of those inputs alone. Hence, $h_T$ is a linear function of $x_t$. Parity (and any phase‑flip automaton) are not linearly representable—therefore mLSTM cannot track the phase.

Empirical evidence in our work:

• We employ only sLSTM blocks because mLSTM "treat[s] sequence elements independently, making it impossible to learn relationships between them directly" (Beck et al. 2024, Sec 2.1).
• Replacing sLSTM with mLSTM, GRU, or vanilla LSTM consistently degrades long‑horizon accuracy, especially on 336‑ and 720‑step forecasts (cf. Sec 4.4).
• xLSTMTime uses sLSTM for smaller forecasting datasets where accurate phase tracking dominates, while mLSTM is chosen only when raw storage capacity outweighs this need (Alharthi & Mahmood 2024). TiRex attributes its performance gains to retaining state tracking as well (Auer et al. 2025).

Thus, sLSTM’s memory‑mixing recurrence preserves and updates time‑varying hidden phases, a property indispensable for high‑fidelity forecasts over long horizons. mLSTM's parallel‑friendly design diminishes this ability. Our empirical results and prior literature confirm that, in forecasting contexts, state‑tracking capacity outweighs raw matrix‑memory capacity—hence our exclusive use of sLSTM blocks in xLSTM‑Mixer. We again thank the reviewer for this question and will integrate this much more extensive theoretical view with illustrative examples in the camera-ready version.

Thank you again for your thoughtful review. We hope this review answered all outstanding questions. Specifically, having elaborated on the theory and limitations. Furthermore, we provided impressive leaderboard placements on the more practical and probabilistic GIFT-Eval set, and discussed and validated additional variate ordering findings. Hence, we have addressed all key concerns regarding the work. In light of this, we would greatly appreciate it if you could raise your score to reflect this.

Best

The Authors

References:

• Liu et al. TimeCMA: Towards LLM-Empowered Multivariate Time Series Forecasting via Cross-Modality Alignment. AAAI, 39(18):18780–18788, 2025.
• Shibo et al. HDT: Hierarchical Discrete Transformer for Multivariate Time Series Forecasting. AAAI, 39(1):746–754, 2025.
• Kong et al. Unlocking the Power of LSTM for Long Term Time Series Forecasting. AAAI, 39(11):11968–11976, 2025.
• Aksu et al. GIFT-Eval: A Benchmark for General Time Series Forecasting Model Evaluation. NeurIPS Workshop on Time Series in the Age of Large Models, 2024.
• Zhang et al. A Survey on Multi-Task Learning. IEEE TKDE, 34(12):5586–5609, 2022.
• Liu et al. iTransformer: Inverted Transformers are Effective for Time Series Forecasting. ICLR, 2024.
• Cao et al. TEMPO: Prompt-based Generative Pre-trained Transformer for Time Series Forecasting. ICLR, 2023.

Dear V19p,

We appreciate the time and consideration you’ve given to our submission.

We hope our detailed replies (to you and the other reviewers) have addressed your original concerns thoroughly. If there are any remaining questions or points that you believe warrant further clarification or discussion, we would be glad to engage further.

Please don’t hesitate to let us know if additional input would be helpful.

Regards,

The Authors

Dear Reviewer 67JL,

We thank the reviewer for recognizing the novel xLSTM-Mixer architecture and its empirical gains over the many baselines in multiple practical scenarios.

We gladly clarify the raised comments and questions:

Re. Clarity: Thank you for thoroughly checking the notation and verifying its correctness. We already clarified the ambiguous pieces and incorporated the changes you suggested. Specifically, we now consistently use the notation introduced in ll. 81ff. This includes the two missing bold notations in Eqs. (1) and (2) and comments regarding Sec. 3.1: There, $\mathbf{x}_t \in \mathbb{R}^V$ intentionally denotes only a single time step of the multivariate time series $\mathbf{X}$, hence the lower-case bold notation introduced before. The definition of $\mathbf{x}^\text{norm}_t$ follows that of the referenced Kim et al. (2022), where $\gamma$ and $\beta$ should also be bold as they are learned separately for each of the $V$ variates. In l. 162, we changed the notation to $\mathbf{h}_0$. Regarding l. 180, we now correctly define $\operatorname{FC}^\text{view}: \mathbb{R}^D \times \mathbb{R}^D \to \mathbb{R}^H$ in the preceeding line. Thank you again for verifying the notation. We are certain this improved the work.

Q1: We aim for best comparability with the existing line of research on long-term time series forecasting—such as the long-term benchmarks evaluated by, for example, Chimera from Behrouz et al. (NeurIPS, 2024), PatchTST from Nie et al. (ICLR, 2023), TiDE from Das et al. (TMLR, 2023), and many preceding works. Therefore, we primarily focused on the benchmark of the classical seven datasets, each evaluated over four forecast horizons, along with an additional 17 datasets presented in the outlooks in Appendices F, G, and H. For the main comparison, we ensured evaluation against the state-of-the-art within each model category (e.g., recurrent, convolutional), which are not necessarily the most recent models. For instance, PatchTST remains the top-performing model in the "deep learning" category of the GIFT-eval leaderboard. Nevertheless, we also include three models from 2025 to reflect the latest advancements in the field.

Q2: Thank you for directing us to GIFT-Eval. The benchmark targets probabilistic zero-shot evaluation, and more than half of its datasets are univariate. Its accompanying pre-training corpus is designed for foundation models rather than classical deep-learning methods, and it focuses on very short horizons, which was initially outside the planned scope of the manuscript. Nevertheless, we ran the benchmark and are pleased to report that xLSTM-Mixer surpasses classical deep-learning and statistical baselines by a large margin on univariate and multivariate datasets, while matching the strongest pre-trained models.

We are grateful for this insightful suggestion, which has broadened the scope of our work and strengthened the manuscript. We will include the full results in the camera-ready version and publish them on the public leaderboard.

Overall Leaderboard

Multivariate Leaderboard Top 10

Univariate Leaderboard Top 10

Q3: We put in substantial effort to ensure a fair comparison. We proceeded as follows: For most models, the initial publication already comprehensively evaluated the model on these well-known datasets and found appropriate hyperparameters. For xLSTMTime, this was not the case, and we were not able to fully reproduce the results in the paper despite our best efforts (cf. ll. 330f). We still present the better results from Alharthi and Mahmood (2024) so as not to erroneously underestimate the method. For the datasets for which no such results were available (ETTh2, ETTm1, and ETTm2), we ran individual hyperparameter searches like we did for our method to ensure a fair comparison between all models.

Q4: We followed the common practice of evaluating the forecast accuracy by counting wins across models and datasets. However, we agree that statistical testing is the more rigorous and desirable procedure to perform this. Following the well-known practice of Demšar (2006), we first perform a Friedman test to ensure the model's performances follow different distributions ($p < 10^{-10}$). Using Holm's method, we can then perform a Conover post-hoc test while adjusting for the family-wise error rate. We must restrict this comparison to one metric and horizon (here, MSE at 96 steps), since the performance results are strongly correlated for one model and dataset pair and would artificially inflate significance if not accounted for.

At a significance threshold of $\alpha = 0.05$, we find that xLSTM-Mixer is statistically significantly better than all other methods, except for xLSTMTime. Yet, the difference in average rank is still impressive, with 1.5 for xLSTM-Mixer and 4.0 for xLSTMTime. Furthermore, xLSTMTime is statistically inseparable from many other models, namely TimeMixer, TSMixer, ModernTCN, Chimera, PatchTST, and TiDE, in that order.

We will add this setup and the result as an easily digestible Critical Difference (CD) diagram to Sec. 4.1 of the paper.

Q5 (curiosity): Thank you so much for the perspective that variate reordering can be seen as ensembling (with weight–sharing). It also sparked our curiosity! We thus performed additional variate experiments with permutations of the variates beyond one (standard order) and two (standard and reversed). Again, using two views works best for all prediction lengths, while further increasing the ensemble size can also be beneficial over the single view. However, this may over-regularize the model for some prediction lengths and higher ensemble counts. We observed this on multiple datasets and show ETTh2 as a representative sample. Thank you again for suggesting this insightful perspective. We will include this in our revised manuscript.

Scaling Summary: ETTh2 (pred_len=96)

Scaling Summary: ETTh2 (pred_len=192)

Scaling Summary: ETTh2 (pred_len=336)

Scaling Summary: ETTh2 (pred_len=720)

Again, thank you very much for this constructive feedback. We hope to have sufficiently answered all questions. In particular, the main concerns, namely the clarity, the statistical significance of the results, and new impressive results on GIFT-Eval, have substantially improved our manuscript. In light of this, we would greatly appreciate it if you reconsidered your score to reflect this.

Best

The Authors

References:

• Behrouz et al. Chimera: Effectively Modeling Multivariate Time Series with 2D State Space Models. NeurIPS, 2024.
• Nie et al. A Time Series is Worth 64 Words: Long-term Forecasting with Transformers. ICLR, 2023.
• Das et al. Long-term Forecasting with TiDE: Time-series Dense Encoder. TMLR, 2023.
• Alharthi et al. xLSTMTime: Long-Term Time Series Forecasting with xLSTM. MDPI AI, 5(3), 2024.
• Demšar. Statistical Comparisons of Classifiers over Multiple Data Sets. JMLR, 7, 2006.

Dear Reviewer 67JL01,

Thank you for the exceptionally thoughtful feedback and for raising your score. Your insights have substantially strengthened our manuscript. We’re integrating your suggestions and have ensured the camera-ready version is typo-free, precise, and fully self-contained.

Bests

The Authors