BRIDGE: Bootstrapping Text to Control Time-Series Generation via Multi-Agent Iterative Optimization and Diffusion Modeling

We sincerely thank the reviewer for the thoughtful and encouraging feedback. We are especially grateful for your recognition of the clarity of the paper, the natural design of the proposed methods, and the strength of our empirical results and discussion. Your comments affirm the value of our effort to introduce a novel task—Text-Controlled Time-Series Generation (TSG)—and the corresponding framework BRIDGE, which combines a multi-agent dataset synthesis pipeline with a hybrid diffusion-based generation model.

Your summary precisely captures the key contributions of our work: (1) the introduction of a new task—Text-Controlled Time-Series Generation (TC-TSG)—to enable semantically guided, controllable synthetic time series; (2) the design of a novel multi-agent framework, which leverages large language models (LLMs) and role-based collaboration for bootstrapping high-quality text-to-time-series paired data; and (3) the proposal of BRIDGE, a hybrid framework that integrates semantic prototypes and text for flexible and faithful time-series generation.

In particular, we are grateful for your acknowledgement that our empirical findings convincingly support both the fidelity and controllability improvements claimed. Your validation that the evaluation criteria and dataset choices are natural and grounded in prior literature is also very helpful. This motivates us to continue refining and extending this line of work. In future iterations, we plan to further investigate theoretical insights behind controllable generation, and explore more efficient multi-agent optimization strategies, building on the foundation established here.

Should you have any suggestions or requests for clarification, we would be more than happy to provide additional details. Thank you again for your constructive and generous evaluation—it is deeply appreciated.

[Q1] What motivated the choice of 16 prototypes, and how does performance change with different prototype counts?

For selecting the optimal number of prototypes, we observe from the ablation study that the generation performance generally improves along with the increase of the number of prototypes (Shown in Table). However, when the number of prototypes is larger than 16, the performance gain is no longer significant. Therefore, we can conclude that the optimal number of prototypes is 16 in our current setting.

MDD Report:

K-L report:

[Q2] How does your work differ from exist agent framework ? Could those be adapted for your task?

MetaGPT follows structured workflows tailored for software engineering tasks, with predefined agent roles and Standard Operating Procedures (SOPs). While effective in that domain, this structure is less readily adaptable to the dynamic, iterative processes needed for generating diverse and semantically aligned text–time-series pairs. AutoGen offers more flexible agent coordination, but lacks built-in support for evaluating alignment between text and time-series data. CAMEL, which focuses on role-playing agents for social simulations via inception prompting, is primarily designed for open-ended tasks rather than modality-aware data construction.

Although these frameworks offer valuable capabilities in their respective domains, adapting them to text-to-time-series generation (TSG) would require substantial modification. In contrast, BRIDGE is purpose-built for TSG, integrating task-specific evaluation modules and iterative refinement mechanisms to support controllability and fidelity.

To better understand their applicability to TSG, we conducted preliminary adaptations of MetaGPT, AutoGen, and CAMEL within our task setting. As shown in the results below, their performance under minimal adaptation was relatively limited. These observations highlight the utility of BRIDGE’s tailored design in addressing the unique demands of TSG, such as modality bridging. Due to space limitations, we only present the MDD results below.

General Comments (GC):

[GC 1] More discussion of privacy risks when generating time series in sensitive domains, such as healthcare.

Thank you for highlighting the potential privacy risks in sensitive domains such as healthcare. While synthetic data is often considered a privacy-preserving alternative to real patient records, there is still a potential risks remain if models overfit and replicate training patterns. Our approach mitigates this risk by focusing on controllable and abstract generation guided by semantic prototypes and textual descriptions, rather than directly replicating raw time-series trajectories. Additionally, our framework does not rely on any personally identifiable attributes—only descriptions of the time series are used during generation. We fully agree that a more in-depth analysis of privacy risks—such as membership inference—would be valuable, and we plan to explore these directions in future work.

---

We are grateful for your engagement and helpful suggestions, which we will carefully incorporate in the revised version.

We truly appreciate the reviewer’s insightful feedback!

[Q1] Justify the use of metrics.

We initially prioritized MDD and KL divergence over PS and DS for their robustness, as they are not influenced by post-hoc models. We have now included PS (MAE reported) and DS and use J-FTSD to complement MSE for controllability evaluation. Our model achieves the best results across all metrics.

PS

DS

J-FTSD

[Q2] Provide more information about semantic prototypes.

Fundamental time series properties for TS generation align with decompositions of TS, such as trends and seasonalities. We use semantic prototypes as latent representations of these commonalities and train the model to decode from combinations of these latent representations for generating certain domain.

The prototype array are fixed after initialization, similar to a initializing a codebook for vector quantization while preventing the codebook from updating along with the decoder model. Instead of updating prototypes, the decoder $\epsilon_\theta$ learns to map them to observed sequences, ensuring implicit alignment with time series semantics.

[Q3] Provide comparisons with Diffusion-TS

Please see [Q 1] for Diffusion-TS results.

[Q4] What motivated the detailed intra-group discussions in the multi-agent setup?

Our design draws from the well-established multi-agent cooperation and competition paradigms, which enhance system performance by encouraging agents to challenge and refine each other’s outputs, mitigating the risk of overconfidence.

Additionally, complex problem-solving is often modeled as programming tasks due to their logical structure. Inspired by this, we structured our system after software development workflows, assigning roles like manager, engineer, scientist, and observer to different agents [1].

[1] LLM-Based Multi-Agent Systems for Software Engineering: Literature Review, Vision and the Road Ahead

[Q5] Use Time-MMD dataset in TS generation

We achieves the best performance on MDD, KL, PS and DS across all subdatasets. Due to space limitations, we only present the MDD results below.

[Q6] Does z correspond to the semantic prototypes?

As stated in Section 3, z is a latent variable sampled from a prior distribution. In the diffusion model, z typically represents random Gaussian noise.

---

Again, we sincerely appreciate the reviewer’s valuable time and insights, which have greatly strengthened our work. Based on your suggestions, we have added PS, DS, and J-FTSD metrics, included Diffusion-TS as a baseline, and expanded experiments on Time-MDD. We also appreciate the recommended references and will discuss them in our revision.

We’d be happy to address any further questions—thank you for your thoughtful feedback and support!

We sincerely appreciate the reviewer’s time and thoughtful feedback. We are grateful for the recognition of the strengths of our work, particularly the “straightforward and well-structured", "reasonable" nature of our approach, as well as the "novel research direction" of BRIDGE in bridging text-controlled generation with time-series synthesis. We also appreciate the acknowledgment of our "innovative" multi-agent dataset synthesis approach.

Your insights are invaluable in helping us refine our work, and we are truly thankful for the opportunity to clarify and improve our paper. Below, we provide our detailed responses to your comments.

General Comments:

It would be better to add computational efficiency and human evaluation of text-to-TS alignment experiments.

We appreciate this suggestion! In fact, we have already included human evaluation of text-to-TS alignment in our original submission. As shown in Table 4 and 5, we compare different configurations (with and without prototypes and text) across multiple datasets using both quantitative metrics (MSE and MAE) and human evaluation scores (HE and HE@3). A detailed discussion of these evaluations can be found in Section 6.2 and 6.5.

For computational efficiency, please refer to [Q 2].

Questions:

[Q 1] How does the multi-agent generated dataset compare to human-annotated text-TS pairs?

In fact, the first step of our multi-agent system (Section 4.1 Step 1) involves collecting articles, news, and reports from the Internet that describe time series data, which serve as human-annotated references ("Initial Text" in Table 1). While these outperform rule-based baselines slightly, they are significantly worse than our multi-agent Refined Text (Table 1), suggesting generic human descriptions are suboptimal for this task.

To address this, we implement a multi-agent iterative optimization process that refines textual descriptions by enhancing key aspects such as trend accuracy, mention of seasonality, completeness of information, and clarity of description (detailed in Appendix A.5). This process strengthens text-to-time series alignment by optimizing both quantitative metrics (e.g., MSE, K-S Test, Wasserstein Distance) and qualitative evaluations based on a 5-point Likert scale. As a result, our refined text-TS pairs achieve significantly higher quality for TS Generation task.

Furthermore, in Section 6.3 Table 1 and Appendix J Table 12, we provide a detailed analysis of what constitutes useful textual descriptions, offering key insights into the generation of high-quality text-TS pairs.

[Q 2] What is the computational cost of BRIDGE relative to prior TS Generation models?

To evaluate cost-efficiency, we measured training time (s/epochs) and inference time (ms/sample) across multiple baselines on same A100 GPU. BRIDGE achieves a favorable balance between performance and cost among diffusion-based models, maintaining high generation quality and controllability with substantially lower computational cost.

[Q 3] Can BRIDGE generalize to unseen domains without fine-tuning? Are there cases where prototype alignment fails?

Our framework was explicitly designed to promote cross-domain generalization by leveraging semantic prototypes and text conditioning. In our experiments (see Table 3), BRIDGE was able to generate high-fidelity time series in unseen domains (e.g., Stock, Web) without any fine-tuning, suggesting that the learned semantic structure and prototype conditioning do generalize beyond the training domains.

While we did not observe clear prototype misalignment in these settings, we agree that such cases may be possible, particularly in domains with very different temporal dynamics or semantics. We will explore this further in future work by evaluating on a broader set of domains and investigating potential prototype refinement strategies.

---

Thank you again for your thoughtful advice. We will certainly incorporate the discussions in the revised version and include references on prompt optimization and LLM-based dataset generation as per your suggestion. Please feel free to raise any further questions—we greatly appreciate your ongoing input!

We appreciate the reviewer's time and feedback, especially the high-level perspective on the entire system, which will guide the future expansion of our work. Thank you!

General Comments (GC):

[GC 1] Claims of generality are not very clear. "The time series specifications (or features) are hand-crafted, and a rich enough set of these is selected."

Section 4.1 generates text templates, which are then applied to TS data, with an LLM filling in general, domain-agnostic statistical features to produce the final text. Thus, time series specifications are automatically generated by the LLM based on templates from the multi-agent system, rather than being manually crafted or selected by human intervention. We will make it more clear in future revisions.

[GC 2] "The extent and impact of human interventions are unclear."

There are only two points in our work where human actions are involved:

1. After collecting the initial text templates (Section 4.1 Step1), LLM prompting filters them to ensure no dataset-specific information is included. A human double-checks the results to prevent potential data leakage.
2. A human assessment (Section 6.2) serves only as an evaluation metric and does not affect model training.

[GC 3] "It isn't clear that existing time series signal decomposition and feature extraction methods couldn't be applied to this data and used for training a generator."

We tried rule-based text generation and Seasonal-Trend Decomposition, but both methods were ineffective in generating useful descriptions (the first paragraph in Section 4 and Appendix A9).

[GC 4] "The contribution to time series analysis and generation is very small."

Our framework is designed specifically to addresses key challenges in text-controlled TS generation, such as lack of aligned data, controllable generation, and domain generalization. We integrate multi-agent reasoning for data construction, semantic prototype conditioning, and a diffusion-based generator to address challenges.

Within the multi agent system, we propose tailored evaluation criteria like trend description accuracy, seasonality, completeness, and clarity (Appendix A.5) to assess TS generation. We also explore the impact of different text types on generation, showing that pattern descriptions and concise summaries are more effective (Section 6.3). These insights offer valuable takeaways for time series tasks.

Other Comments (OC):

[OC 1] "It would be very helpful to clearly show the hand-tuned and human interaction portions."

We clarify that there is no hand-tuning or human intervention for time series features.

Please refer to our response to [GC 1] and [GC 2].

[OC 2] "Can this framework be applied to other applications?"

Great suggestion! The modular structure could support future applications beyond TS Generation, including in image and video. This is an exciting direction for future work.

[OC 3] “Why is the multi-agent approach a good one? Isn't this just an optimization problem? What other optimizers could be applied?”

We chose a multi-agent system over a single-agent or LLM prompt approach for key reasons. Direct LLM prompting was ineffective in generating useful descriptions (see [GC 3]), and single-agent systems often suffer from overconfidence, leading to errors without correction [1]. Multi-agent systems allow agents to critique each other’s outputs, improving reliability and transparency by assigning distinct roles.

Our multi-agent approach goes beyond scalar optimization, involving tasks like searching for TS text online, summarizing templates, generating text, evaluating alignment, and refining for better templates.

While we recognize reinforcement learning could optimize a single LLM, it faces challenges: (1) large optimization space, as RL starts from scratch, while our method refines existing templates; (2) high computational demands due to model training, whereas our agents require no retraining or fine-tuning; and (3) sparse feedback from using final generation performance as a reward, while our agents themselves evaluate text clarity, completeness etc (Appendix A5), improving refinement efficiency.

[1] Enhancing LLM Reasoning with Multi-Path Collaborative Reactive and Reflection agents [arxiv.org/pdf/2501.00430]

Question:

“Given the hand crafted selection of time series specifications, then why is the LLM-based approach a good one?”

Please kindly refer to our response to [GC 1], [GC 2] for the clarification on avoiding human intervention in feature selection and [OC 3] for justification of the multi-agent system. As general statistics with simple rule-based texts are ineffective for TSG (see Section 6.3), we use a multi-agent system.

If any further questions or concerns, we’d be happy to provide more information. Looking forward to your response and continued discussions!