For W1

Ablation experiments During the task execution phase, our goal is to leverage agentic framework to drive small models in completing various trajectory modeling tasks that LLMs are not good at, while further optimizing the performance of small models. We compared the effects of independent execution and TrajAgent joint execution, with the experimental results presented in Table 1. In the table, "-DL" represents our implementation method designed with reference to other large model approaches, "-LLM" denotes the small model, and "Joint" refers to the result of large models driving small models with optimization. MAE=100% and Acc@5=0 values in the table suggest that no effective prompt-based LLM approaches were found for these tasks.

Figure Thank you for your suggestions regarding the figures. We apologize for omitting some arrows in Figure 1. The direction of the arrows aligns with the flow of data/information, originating from the raw data (Data module) and input into the TrajAgent framework (Framework module). The large model automatically performs data preprocessing, small model selection, and small model deployment training based on the input data type and task description, then dynamically selects the optimal optimization strategy based on the training results, ultimately producing an experimental solution applicable to various downstream tasks (Task module). The left side of the dashed line represents the traditional single-model paradigm, where a single model is designed for a specific task and is only suitable for particular datasets. The right side of the dashed line represents our solution, which constructs a unified trajectory modeling framework adaptable to multiple tasks and datasets. In Figure 2, the colored blocks represent the currently integrated models for each type of task. Since the models and datasets will be further expanded in the future, the names of each model are not labeled.

Table 1. Ablation experiments

For W2

Thank you for your suggestion. We have integrated the DutyTTE model you proposed. For the MoEUQ method, the optimization results are shown in Table 2. TrajAgent autonomously selects key parameters and configurations.

Table 2. Expanding models and optimization methods

For W3

Thank you for your reminder and suggestions. We apologize for not updating the information of the cited articles in a timely manner, which was an oversight during the resubmission process. We have updated this information in the latest version of the paper, along with the recent work mentioned in the "tiny weakness" section.

For W4

Thank you for your question!

• Extended Models For the LLM-based method (e.g., LLM-ZS), since it uses a pre-trained model and the trajectory data does not participate in training, no data augmentation (DA) method is provided. We allowed TrajAgent to automatically optimize the key parameters mentioned in the original paper, and the results are shown in Table 2. For other small models using map/GPS trajectory data(e.g., GraphMM), we adopt the rule-based data augmentation approach from DeepMM, which first down-samples the raw data, then designs various noise-injection rules, and finally combines the processed data with the original dataset to achieve data augmentation. We integrated it into the TrajAgent framework, using agent to choose noise-injection strategies. The experimental results are shown in Table 2 (base_model=llama3-70b, step=5).

• The optimization effect diminishes as the step increases

  While selecting combinations of operators, the model further optimizes the configuration of each operator (e.g., the original configuration file for "inserter" is {insert_nums: 1, insert_ratio: 0, insert_time_sort: maximum, percent_no_augment: 0, ti_insert_n_times: 1}). In a zero-shot scenario, the model explores optimization strategies based on dataset characteristics and the meaning of the operators, with a probability of converging to a local optimum—i.e., finding a suboptimal combination and deeming the result sufficient, thus stopping the exploration of new combinations and selecting the best operator configuration based on this combination. We compared the llama3-70b used in the paper with other reasoning models, and the results are shown in Table 3 (S2TUL, FSK-London, memory-length=1). We found that models with stronger reasoning capabilities attempt more operator combinations and have a higher probability of finding better combinations in fewer steps. For instance, DeepSeek-v3 outperforms LLaMA-3-70B in step 4. Under the same operator combination and the same number of steps, models with stronger reasoning capabilities yield better optimization results. For instance, DeepSeek-v3 outperforms LLaMA-3-70B in steps 3, 5, 6, 7, 9, 10, 11, and 13.

  • Improvement solution

  Implement reflection similar to contrastive learning between steps, such as further adjusting the parameters of each operator based on effective combinations, to avoid exploring ineffective combinations as much as possible. The improved results are shown in Table 4 (S2TUL, FSK-London, memory-length=1).

• The optimization effect diminishes as the memory_length increases

  Memory_length refers to the number of action proposals generated in each reasoning step. Increasing it can improve exploration efficiency but also raises the probability of falling into a local optimum. We compared the performance of llama3-70b with that of reasoning models, and the results are shown in Table 5 (S2TUL, FSK-London, step=5). Both models achieved relatively good results at memory_length=2, but as the length increased, they faced the issue of converging to suboptimal solutions. However, models with stronger reasoning capabilities exhibited a stronger tendency to explore other combinations, thus having a higher probability of finding better combinations and escaping the "optimization trap" .

  • Improvement solution

  Periodically optimize and update the memory organization, discard poor combinations, retain good ones, and guide the model to explore new combinations during the reflection phase. The improved results are shown in Table 6 (S2TUL, FSK-London, step=5).
  | base-model| step | 0| 1 | 2 | 3 | 4 | 5| 6| 7| 8 | 9| 10| 11| 12| 13| 14| 15| 16| 17| |-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-| | llama3-70b | ACC@5| 56.28 | 56.28 | 59.10 | 68.18 | 68.18 | 68.39 | 68.18 | 69.04 | 69.04 | 68.39 | 69.04 | 69.04 | 87.01 | 69.05 | 68.40 | 68.19 | 83.12 | 88.96 | | deepseek-v3 | ACC@5| 56.28 | 79.22 | 59.52 | 61.26 | 81.17 | 68.83 | 69.26 | 69.05 | 83.98 | 70.35 | 69.05 | 69.26 | 71.00 | 61.90 | 60.82 | 68.83 | 71.00 | 66.88 | | gemini-2.0-flash-001 | ACC@5| 56.28 | 73.16 | 61.69 | 70.35 | 68.83 | 82.25 | 69.05 | 68.40 | 87.01 | 65.58 | 79.87 | 83.12 | 71.65 | 84.63 | 82.47 | 71.43 | 69.05 | 69.05 | Table 3. Comparative analysis of TrajAgent performance across optimization steps

We sincerely appreciate your valuable questions. For your convenience, we have summarized the questions that were previously simplified-away in the main response or contained in other reviewers' responses impacted by length constraints. We would be grateful if you could kindly confirm whether it satisfactorily resolves your concerns. Should the explanations meet your expectations, we would deeply appreciate your consideration for score improvement. If any uncertainties remain, we are fully prepared to provide additional clarifications through further discussion to eliminate any doubts and collectively enhance the paper's quality.

Complete versions of Tables 4 to 6

The "detailed" column specifies the selection sequence of data augmentation operators, where each numeric code represents a specific operator. For example, [1, 2, 9] indicates that the data should undergo crop, insert_unvisited, and reorder operations sequentially. Additionally, TrajAgent needs to configure detailed parameters for each operator, such as:

crop: {crop_n_times: 2, crop_nums: 2, crop_ratio: 0, crop_time_sort: minimum, ti_crop_n_times: 3}

After improvement, the model can efficiently explore better operator combinations and further select optimal configurations.

For tiny weakness

1. We apologize for this oversight. In Eq. (1), T stands for Task. TrajAgent parses the corresponding trajectory modeling task based on user requests, selects the appropriate trajectory dataset according to the request and task type, and then chooses the suitable small model based on the dataset and task.
2. We have supplemented our references with additional studies employing similar architectures, including agentic framework for single task(e.g., MobAgent[2]) and autonomously optimizing Agentic AI solutions(e.g., ADAS[1])
3. Thank you for pointing out the spelling errors and missing related work. We have reviewed the entire article and revised these issues based on your suggestions.

References

[1]Wang Y, Chen Y, Zhong F, et al. Simulating human-like daily activities with desire-driven autonomy[J]. ICLR 2025. [2]Hu S, Lu C, Clune J. Automated design of agentic systems[J].ICLR 2025.

For W1

Thank you for your question! We apologize for not elaborating in detail in the paper. Our key innovations are:

1. Compared to other works that use LLMs and fixed-parameter small models to solve urban science problems (e.g., UrbanLLM), we support online training and inference for small models, using real-time inference results as feedback in the agent’s reasoning loop, leading to more reliable outcomes.
2. Compared to other agent frameworks that automate task execution (e.g., HuggingGPT), we introduce an online optimization mechanism on top of automation.
   For check-in trajectory data, we explore augmentation from the perspective of time intervals, defining ten operators (cropper, inserter, masker, slide window, etc.), each with configurable parameters. Instead of rule-based search, we use large models + evolutionary algorithms to optimize the memory pool, searching for the best operator combinations, sequencing, and parameter settings of each operator, while adjusting strategies based on training feedback. Experiments show that, unlike sequential recommendation models (e.g., CoSeRec[6]) perform well in common augmentation methods, different trajectory modeling model architectures require different augmentation strategies depending on the dataset (see Table 3).
   For map/GPS trajectory data, we explore downsampling followed by noise injection (Please refer to GraphMM in following Table).
3. Compared to traditional optimization tools (e.g., Optuna[9]), we use LLM agents to retrieve key parameters and infer optimal augmentation strategies based on prior knowledge and online training results. We also explore combining data augmentation with parameter optimization to expand the search space and improve performance

For W2

Thank you for your valuable question—this is indeed part of our ongoing and future work. For the most common issues, solutions, and experimental results, please refer to PauW’s W3.

For W3

We apologize for not providing sufficient details in the paper.

• What types of feedback are most valuable

  The model treats optimization strategies (e.g., operator combinations, parameter configurations) and their corresponding training results as feedback.

• How does the agent adapt during iteration

  The framework sets a dynamic threshold that increases over time. Feedback with training results above the threshold is stored as good memory, while the rest is stored as bad memory. The framework periodically updates the memory pool, discarding bad memories and retaining good ones. During the reflection phase, inspired by contrastive learning, the model is guided to learn from good memories and avoid bad ones.

For Q1

LLMs inherently possess knowledge of deep learning and data engineering. Prior work has shown that LLMs can perform data processing based on textual descriptions[7].

Within the TrajAgent framework, LLM agents perform data preprocessing and model selection by analyzing dataset descriptions, samples, task requirements, and model specifications from abstractions of papers, and dynamically adjusting optimization strategy for operator selection and parameter configuration based on real-time feedback from small model training, with all model deployment and training executed through automated shell script calls.

For Q2

Thank you for your question. To integrate a new model, simply add Task description , data description , model description and codes of the model (as mentioned in Q1) to the corresponding part of the TrajAgent framework. For the agent’s reasoning process, refer to du8A’s W3. Due to space constraints, we could not list all details in Table 2, but we can provide them in the subsequent phase.

For Q3

We apologize for the lack of clarity in the paper. In Table 1 of original paper, the difference between Origin and +JO reflects TrajAgent’s optimization effect.

Additionally, as shown in Figure 1 , the left side (dashed line) represents traditional small models, which often handle only single tasks or perform poorly across multiple tasks. In contrast, TrajAgent can handle multiple tasks and automatically select the best small model for each task.

For Q4

Thank you for this practical question. In Table 7, we provide detailed token consumption when TrajAgent uses Large model methods and Deep learning methods.

As mentioned in Q2, TrajAgent’s framework can be easily extended to tasks/domains not yet integrated into UniEnv, which is part of our future work.

For W1

We appreciate your feedback and apologize for the oversight in our writing. In Eq. (1), T denotes the Task. The TrajAgent first parses the trajectory modeling task from user requests, then selects an appropriate trajectory dataset based on both the request and task type, and finally chooses a suitable small model according to the dataset and task requirements.

For W2

We regret not clarifying this in the paper. As illustrated in Figure 5, each functional module is executed by an agent in sequential order:

1. Task Parsing
2. Model Selection
3. Data Processing
4. Task Planning & Execution (including data augmentation and parameter optimization)
5. Task Summarization

As a result, there are six agents in total.

For W3

We apologize for the insufficient explanation in the manuscript. Within the TrajAgent framework, the agents perform data preprocessing and model selection by analyzing: dataset descriptions, dataset samples, task descriptions, and model descriptions extracted from original paper abstracts, along with natural language instructions. It can retry or adjust selections based on executing results of modules. Optimization strategies (operator selection/sequencing, parameter tuning) are formulated by analysing parameter descriptions, model descriptions, dataset descriptions, and operator specifications. The agent dynamically refines strategies based on real-time training results from small models.

Background information (dataset/task descriptions etc.) is provided for the THOUGHT phase, where the agent analyzes information and stores plans in short-term memory. During ACTION, it executes configurations by writing parameters to model/operator config files. Model deployment/training is executed via shell scripts, with results fed back to the framework. The system maintains an adaptive threshold to classify results into "good" (stored in long-term memory) or "bad" memories. Regular memory optimization purges bad memories while retaining good ones. Periodic REFLECTION phases allow the agent to learn from good results and avoid bad results, extracting experiential knowledge to guide subsequent THOUGHT processes.

For W4

We sincerely thank you for this valuable question. For supplemental responses, please refer to MyNr's W2. Our approach can be compared with three agentic method categories:

1. Automatic workflow optimizers (e.g., AFlow, AgentSquare[8]): These focus on public benchmark problems with deterministic solutions(e.g., GSM8K), making them unsuitable for trajectory modeling tasks.
2. Domain-specific problem solvers (e.g., WebAgent[9]): While effective for targeted applications like web search, they lack transferability to other domains.
3. Urban science models (e.g., UrbanLLM): Though pretrained for urban tasks, they cannot perform online training/inference or further optimize results.

For W1

We appreciate your questions and apologize for not providing sufficient details in our paper.

1. Data Augmentation:

• We augment check-in trajectory data by considering temporal intervals and define ten operators (e.g., cropper, inserter, masker, slide window), each with configurable parameters. We employ LLM-guided optimization combined with an evolving memory pool to identify the best strategy, including operator combinations, sequencing, and parameter settings (Please refer to PauW's W4). The strategy is dynamically adjusted based on training feedback.
• Experiments show that, unlike sequential recommendation models (e.g., CoSeRec) perform well in common augmentation methods, different trajectory modeling architectures require tailored augmentation strategies depending on the dataset (Please refer to Table 3 in MyNr's W1).
• For map/GPS trajectory data, we adopt DeepMM's rule-based approach: first downsampling the raw data, then applying noise injection rules, and finally merging the augmented data with the original (Please refer to GraphMM in Table 2 in PauW's W2).

2. Parameter Optimization:

   We leverage LLMs to autonomously search for critical parameters and infer optimal configurations based on real-time training feedback.

3. Joint Optimization:

   By combining data augmentation with parameter tuning, we expand the optimization space, leading to further performance gains.

This integrated approach enables adaptive, task-aware optimization while maintaining computational efficiency. We will incorporate these clarifications in the revised manuscript.

For W2

Thank you for raising this critical issue. Table 7(Please refer to MyNr's Q4) compares token usage between LLM methods (includes inherent LLM overhead) and Deep learning methods (pure TrajAgent consumption)
Note: Consumption scales with step count and memory_length.

For W3

Our experiments followed zero-shot settings (see PauW's W4), allowing free exploration of operator/parameter combinations. We experimentally removed the scoring and evaluation annotations from each record in the long-term memory, which prevented the agent from distinguishing between good and bad records during its reasoning and reflection phases, thereby impairing its ability to effectively learn from successful cases. As shown in Table 8, the results demonstrate that in our original design, TrajAgent's performance consistently improved with increasing steps and memory size (improvement strategies are detailed in PauW's W4). However, when memory scoring was removed, the performance failed to show progressive enhancement despite the growing number of steps.

For W4

We appreciate your feedback! We apologize for not clarifying this in the paper - all our experiments were conducted under zero-shot settings to ensure the model can freely explore optimal solutions based on current conditions and fully leverage its reasoning capabilities (for related discussions, please refer to MyNr's Q2). The learning mechanism of our agent framework resembles the trial-and-error approach in RL, which can generalize to unseen tasks or new domains not covered by UniEnv.

For W5

Thanks for your valuable feedback. Addressing this issue is one of our most recent research efforts. A detailed analysis of the problem's root causes, proposed solutions, and corresponding experimental results can be found in PauW's W4.

References

[1]Beneduce C, Lepri B, Luca M. Large language models are zero-shot next location predictors[J]. IEEE Access, 2025.

[2]Liu Y, Ge Q, Luo W, et al. Graphmm: Graph-based vehicular map matching by leveraging trajectory and road correlations[J]. IEEE Transactions on Knowledge and Data Engineering, 2023, 36(1): 184-198.

[3]Feng J, Li Y, Zhao K, et al. DeepMM: Deep learning based map matching with data augmentation[J]. IEEE Transactions on Mobile Computing, 2020, 21(7): 2372-2384.

[4]Wang Y, Chen Y, Zhong F, et al. Simulating human-like daily activities with desire-driven autonomy[J]. ICLR 2025.

[5]Hu S, Lu C, Clune J. Automated design of agentic systems[J].ICLR 2025.

[6]Liu Z, Chen Y, Li J, et al. Contrastive self-supervised sequential recommendation with robust augmentation[J]. arXiv preprint arXiv:2108.06479, 2021.

[7]Gandhi S, Gala R, Viswanathan V, et al. Better synthetic data by retrieving and transforming existing datasets[J]. ACL 2024 Findings.

[8]Shang Y, Li Y, Zhao K, et al. Agentsquare: Automatic llm agent search in modular design space[J]. ICLR 2025.

[9]Gur I, Furuta H, Huang A, et al. A real-world webagent with planning, long context understanding, and program synthesis[J]. arXiv preprint arXiv:2307.12856, 2023.