LangTime: A Language-Guided Unified Model for Time Series Forecasting with Proximal Policy Optimization

We sincerely thank you for the thorough review and insightful comments. In our response, our model was jointly pre-trained on the ETTh1 and Weather datasets, and experiments involving the TimePPO stage were fine-tuned on individual datasets. We presented the average of the results, and detailed experimental results are available at link.

Suggestions: Figure 2 contains too much compressed information, making it difficult to follow.

Thank you for your suggestion. We have re-drawn Figure 2 and presented it as Figure 1 in the link. In the next version, we will update some images in the paper for clearer expression.

W1: Do not mention the training efficiency, especially with autoregressive output approach.

Our model achieved the results in the paper with just 1 epoch of pre-training, partially compensating for the slower training speed. Due to the complex procedure of the PPO algorithm, the TimePPO fine-tuning phase is slow. However, our experiments demonstrate that fine-tuning with limited data (Tab.6 in our response to reviewer 5obH's Q4) and mostly frozen parameters (Tab.2) still yields good performance, enhancing its scalability under resource constraints.

Table 1 Training Speeds of Different Models

Table 2 Impact of Fine-tuning Data Volume on ETTh1

W2: The pre-training process is unclear.

We pre-trained on 7 datasets (ETT, Weather, Exchange, Electricity) and conducted zero-shot experiments on two unseen datasets (Traffic, Illness). All models in Table 5 in paper were trained under the same dataset settings, making the comparison fair.

Q1: Is there a specific reason for selecting Qwen2-0.5B-Instruction as the backbone LLM?

Compared to GPT2, Qwen is pre-trained on larger datasets, offering better language understanding and stronger instruction-following abilities to better guide LLM in interpreting time series. While the common Llama-7b also has powerful capabilities, its large number of parameters led us to choose the smaller Qwen2-0.5B-Instruction model as the backbone. In Table x, we compared different backbones; GPT2's performance on Weather is close to Qwen, demonstrating our framework's effectiveness, but its performance on ETTh1 reveals its limitations.

Table 3 Performance of Different Backbones

We sincerely thank you for the thorough review and insightful comments. In our response, models were jointly pre-trained on the ETTh1 and Weather, and experiments involving the TimePPO stage were fine-tuned on individual datasets. We presented the average of the results, and full results are available at link.

E1: Hyperparameter Explanation

For Eq. (7), we introduce parameter ξ based on GAE [1] to address error accumulation effects. When prediction steps increase, model errors often exceed those of repeating previous steps, causing underestimated advantages. The coefficient ξ(<1) relaxes this constraint to better reflect relative prediction quality. For the hyperparameters in Eq.(6) and Eq.(8), $\beta$ controls the MSE penalty term, aiming to regulate the extent of policy updates from the reward score perspective. For $\eta$, we referenced the alignment tax design in InstructGPT, enhancing the model's prediction capability and training stability. Sensitivity analysis experiments showed no significant impact.

Tab.1: sensitivity analysis for $\beta$

Tab.2: sensitivity analysis for $\eta$

A detailed parameter discussion will be added in the paper's next version.

[1] Schulman, J. et al. High-dimensional continuous control using generalized advantage estimation. ICLR2016.

E2: Performance Across Backbones

The rationale for selecting Qwen and experimental results with other LLMs are detailed in Q1 of our response to Reviewer bFpj, demonstrating adaptability across LLMs and GPT2's limitations.

W1: Training Efficiency Discussion

For computational efficiency concerns, please refer to our response W1 to Reviewer bFpj.

Q1: Difference Between Language Prefixes and Language-Guided Strategy

Using language as prefixes inadequately describes the cross-modal relationship between language and time series, hindering LLMs' comprehension of unseen time series data during pre-training and causing modality misalignment. LangTime provides equivalent domain information while guiding LLMs through task-specific instructions (compression and prediction) and dual training objectives (reconstruction and prediction) to achieve alignment. For additional details, see our response to Q2 from Reviewer BdAz. To validate LangTime's effectiveness in enhancing LLMs' time series understanding, we replaced the LLM with a linear layer. As shown in Tab.3, the significant performance decline confirms the necessity of our language-guided strategy.

Tab.3: Impact of Removing LLM

Q2:Could the authors explain why these specific token forms were chosen and what advantages they offer over alternative tokenization strategies?

The special tokens <|EMB|> and <|OUT|> are newly added learnable tokens without predefined meanings for LLMs. Their specific forms were chosen primarily for human readability rather than offering inherent advantages in modality alignment. To verify this, we conducted experiments with alternative tokens (<|ABC|> and <|DEF|>), which showed no significant performance differences(Tab.4).

Tab.4: Impact of Token Replacement

Q3: How does the choice of mask rate impact the model's ability to extract temporal patterns and maintain prediction accuracy over longer horizons?

The mask rate balances two objectives: (1) preventing overfitting in datasets with varying convergence speeds, and (2) enhancing temporal pattern learning through reconstruction tasks. Experiments show that extreme mask rates (too low/high) degrade performance: low rates fail to improve temporal pattern extraction, while high rates impair long-horizon predictions. Shorter prediction lengths suffer more from error accumulation(Tab.5).

Tab.5: Impact of Mask Rate and Prediction Length on ETTh1

Q4: Does TimePPO fine-tune all parameters of the model, or are specific parts of the model frozen during fine-tuning?

In our experiments, TimePPO fine-tunes full parameters. However, freezing certain components still achieves similar performance, demonstrating the method's scalability.

Tab.6: Effect of Fine-tuning Parameters

We sincerely thank you for the thorough review and insightful comments. In our response, our model was jointly pre-trained on the ETTh1 and Weather datasets, and experiments involving the TimePPO stage were fine-tuned on individual datasets. We presented the average of the results, and detailed experimental results are available at https://anonymous.4open.science/r/full-E4EE/README.md.

Q1:Could you provide a detailed explanation of the prompt construction process in your approach, and further discuss the model’s generalizability when faced with slight variations in the prompt formulations?

The prompts in our approach consist of two parts. The first part includes common domain descriptive information used in existing methods to provide richer linguistic information. The second part has two task instructions to guide the model in understanding the time series data and generating predictions. We compared the impact on performance when these two parts varied separately, as shown in Table 1. However, when their expression changes but the basic meaning remains consistent, modifying the domain description and instructions does not significantly affect the model's performance.

Table 1: Impact of language prompt modification on model capability

In Table 1, Instruction Prompt means changing TCP to the following format:

The details of the provided time series: 
Period: <Timestamp>,
Dataset: <Dataset Information>,
Channel: <Channel Information>,
Value: <Time Series Representation>,
Please summarize this series in a single term: <|EMB|>. 
Using the given details, forecast the upcoming <N> values: <|OUT|>

Data description Prompt means modifying the descriptive information of the following two datasets:

ETTh1: {
Original: "An hourly-sampled electricity transformer dataset intended for electrical asset monitoring, collected from one area in a province in China."
Modified: "An hourly-sampled dataset of electricity transformers designed for monitoring electrical assets."
},
Weather: {
Original: "Meteorological indicator data with ten minute sample rate."
Modified: "Data on meteorological indicators sampled every ten minutes."
}

Q2: In terms of aligning language with time series data, what are the key innovations of your method compared to existing approaches such as TimeLLM, UniTime, and MetaTST?

1. Characteristics of Existing Methods: Existing approaches like TimeLLM and UniTime integrate linguistic information with time series data merely by concatenation, failing to emphasize the relationship between the two parts. This can make it challenging for LLMs, which have not encountered time series data during pre-training, to understand sequential correlations. MetaTST, on the other hand, does not employ LLMs as its backbone architecture, therefore it does not leverage LLMs' comprehension capabilities for time series data. While these methods leverage the comprehensive pre-trained knowledge of LLMs by providing linguistic information, they overlook the powerful instruction-following abilities of LLMs.

2. Innovations of Our Method: Our approach introduces diverse domain information via TCP and utilizes two directives to present LLMs with dual tasks: first, compress the presented information to generate a model understanding; then predict future outcomes based on this understanding (compressed token) following instructions. Benefiting from LLMs' robust instruction-following capacity, our language-guided approach aids LLMs in comprehending time series data by integrating reconstruction and prediction training objectives to fully exploit LLM's time series understanding and prediction abilities. Additionally, LangTime employs an autoregressive structure, allowing flexible prediction length. Inspired by RLHF and considering time series data characteristics, we designed a reward function with multi-angle assessment and a repetition strategy-based Value function. Fine-tuned via PPO algorithm, this enhances the model's ability to combat cumulative errors.

To validate our method's effectiveness in enhancing LLMs' time series understanding, we replaced the LLM with a linear layer. As shown in Table 2, the significant performance decline confirms the necessity of our language-guided strategy.

Table 2: Impact of Removing LLM

As for Eq.(8), it should actually be: $-||y-\hat{y}||^2_2$ (refer to actor_loss_fn in ppo_trainer.py). We will modify the description of hyperparameters and Eq.(6) and Eq.(8) in the next version of the paper. Full results are available https://anonymous.4open.science/r/full-E4EE/README.md.

C1: Comparison of LangTime SFT with UniTime SFT

Unlike UniTime and TimeLLM, LangTime uses a more flexible autoregressive structure. SFT focuses on the process of generating the next prediction, which has limited effectiveness on the continuous generation process of autoregressive models. This explains why LangTime performs poorly during SFT.

Q1: Cross-Domain Generalization

We need to clarify that LangTime undergoes joint pre-training on 7 datasets, which enables it to understand time series across different domains. When facing unseen domains, we provide description information for each channel in the new dataset via TCP. Leveraging the rich language knowledge of LLM and the understanding of time series across different domains acquired during pre-training, LangTime addresses the cross-domain generalization issue. We demonstrate its generalization capability in Table 5 of the paper. We conducted pre-training on ETTh1 and ETTm1, and evaluated on 3 other datasets.

Tab.1: Cross-domain Generalization Ability

Q2: How does the TCP used in this paper differ from the language prompts used in UniTime and TimeLLM?

The prompt-as-prefix approach (UniTime/TimeLLM) cannot fully describe the connection between the language part and the time series part, making it difficult for the LLM to understand time series data that it has never seen during the pre-training, leading to challenges in modality alignment. LangTime provides the same domain information as the prompt-as-prefix approach while guiding the LLM to compress and predict time series through two task instructions. Modality alignment is achieved through reconstruction and prediction tasks. You can also refer to our response to reviewer BdAz's Q2 for more details.

Q3: Effects of $\beta$ and $\eta$

Due to length constraints, please refer to our response to reviewer 5obH's E1.

Q4: Comparison of LangTime against foundation model

Because TimesFM is pre-trained on a large number of datasets, to maintain the invisibility of the test domain, we re-trained LangTime on the Weather, ECL, and Exchange datasets, and conducted zero-shot testing on the ETT dataset. As shown in Tab.2, LangTime achieved better performance on most datasets, demonstrating its effectiveness in cross-domain generalization.

Tab.2: Zero-shot performance comparison of LangTime and foundation model

Q5: Why does TimePPO specifically contribute towards reducing error accumulation?

Autoregressive models have flexible prediction lengths but are significantly affected by accumulated errors because they lack the ability to discern whether the output of the previous step is reliable. TimePPO estimates the return for the entire sequence through the Value Function and evaluates the long-term value of the current step relative to the estimated level using advantages. These designs optimize the prediction of the entire sequence rather than just focusing on the model's ability to predict the next step. Thus, TimePPO helps alleviate the cumulative error issue in autoregressive models. To verify this, we compared the model's metrics of the last step in long-term predictions (336, 720). As shown in Tab.3, TimePPO performs better in the final step predictions most affected by accumulated errors.

Tab.3: TimePPO'role in error accumulation

Q6: How does the output sequence length affect the error accumulation?

The output sequence length used in this paper is 96 (the prediction length for each step in autoregression). Since errors occur during each prediction, they accumulate continuously during iterations. If the single output sequence length is small, more steps are needed to predict the same length, making accumulation errors more pronounced and prediction performance relatively poorer. Conversely, although reducing the number of steps, predicting a longer sequence in a single prediction may lead to worse results. To verify this, we compared the impact of different single prediction lengths on final performance.

Tab.4: Impact of single prediction length