UniTraj: Learning a Universal Trajectory Foundation Model from Billion-Scale Worldwide Traces

W1

We appreciate the reviewer’s thoughtful observation regarding potential geographic imbalance in the WorldTrace dataset. We agree that ensuring global representativeness is crucial for universal modeling. As acknowledged in Appendix E.1 (Limitations), some regions are less represented in WorldTrace due to public data availability and the current state of community-contributed GPS traces.

Despite these imbalances, UniTraj still demonstrates remarkable generalization across a variety of regions, as evidenced by:

1. As shown in Table 2 and Table 3, UniTraj performs strongly even in regions outside the dominant geographic areas, like Grab-Posisi (from Southeast Asia), contributors of WorldTrace.
2. In zero-shot settings, UniTraj shows notable robustness on these datasets, demonstrating its cross-regional generalization capacity, even with partial underrepresentation.
3. The self-supervised pretraining strategies (e.g., ATR and STM) were designed explicitly to mitigate region-specific overfitting by forcing the model to learn from varying sampling rates, motion patterns, and missing data.

W2

We thank the reviewer for highlighting this critical design trade-off. Indeed, our decision to omit contextual features such as road networks and POIs was a deliberate and principled choice to align with our core objective: developing a globally deployable and region-independent trajectory foundation model. Our core motivation for acting in this manner is as follows:

1. Many prior models rely on region-specific information (e.g., local road topologies, POIs, and semantics), which hinders transferability across cities or countries due to significant infrastructure, semantics, and data availability differences.
2. Our goal was to train a universal backbone to generalize across diverse geographies and support plug-and-play adaptation without requiring aligned contextual data from every target region. As noted in Lines 240–245 of the main paper, this decision helps UniTraj avoid hard-coded dependencies and enables zero-shot generalization to the unseen areas.

Despite this minimalist design, UniTraj achieves superior performance in zero-shot and fine-tuning settings across diverse datasets and tasks (see Tables 2 & 3, Figure 2). This demonstrates that the model effectively captures complex movement patterns and spatio-temporal dependencies using only core trajectory signals. It also generalizes robustly to unseen regions, which would not be feasible if region-dependent features were required.

W3

Thank you for your constructive feedback and for highlighting the need to clarify our reasoning behind the asymmetric encoder-decoder structure in UniTraj.

• Motivation: Our objective was to balance model expressiveness with computational efficiency during pre-training on large-scale trajectory data. The encoder is tasked with learning rich representations from observed trajectory segments, and thus we allocate more layers and capacity to it. The decoder is used only during pre-training to reconstruct the masked trajectory points. During downstream tasks, the decoder is discarded, and only the encoder is used. This design enables efficient fine-tuning and inference.
• Empirical Support: We conducted model ablation and parameter sensitivity experiments (Appendix D.4, Figure 3c), which show that increasing the encoder depth improves representation quality up to a saturation point, while deeper decoders provide diminishing returns during reconstruction. This design reduces overfitting risk and improves generalization.

Q1

Thank you for your insightful question regarding the masking design. The masking ratio combination was determined empirically through a grid search on the validation set using the WorldTrace and Chengdu datasets. We observed that prioritizing random masking (50%) provided the strongest generalization and robustness for local and global dependencies, while the remaining ratios introduced targeted challenges relevant to specific downstream tasks.

Results in Section 5.4 (see Figure 3(d)) show that model performance is moderately sensitive to the masking ratio. Using less than 50% random masking or omitting certain masking types degraded performance, while the chosen combination achieved the best balance between reconstruction accuracy and transferability.

Q2

Thank you for highlighting the importance of discussing RoPE.

We selected RoPE over other positional encoding schemes for the following specific reasons:

• Relative Position Awareness: Unlike sinusoidal or absolute positional embeddings, RoPE encodes relative distances between points via rotation in embedding space, which is critical for trajectory data where relative motion (e.g., turns, acceleration) matters more than absolute position indices.

• Continuous and Scalable: RoPE naturally supports continuous and extrapolatable temporal sequences, making it better suited for modeling variable-length trajectories without discrete position limits or index overflow.

Q3

We are very grateful to the reviewers for their meticulous reading and valuable insights. The higher errors on low-quality datasets such as GeoLife and Grab-Posisi are primarily due to their irregular and sparse sampling intervals, higher levels of GPS noise, and greater diversity in travel modes. Qualitative analysis of failure cases reveals two main patterns:

• Infrequent or uneven sampling often leads to large spatial gaps between points, making it challenging for the model to accurately reconstruct complex maneuvers (e.g., sharp turns or mode switches) that occur between observations.
• Poor GPS quality and inconsistent data can introduce outliers or implausible points, which may mislead the model during both pretraining and recovery, resulting in larger reconstruction errors.

In addition, from our ablation studies (Table 4), we observed that removing dynamic resampling or key point masking causes sharp degradation on these datasets, indicating that temporal heterogeneity and spatial-level geometry are major failure modes. In some failure cases, the model over-smooths short, abrupt turns or fails to reconstruct loops or detours, especially when large sections are masked.

Thank you for the author's detailed response. I am very satisfied with the author's clarification, especially the thorough analysis of real-world scenarios in Q3. I suggest the authors integrate the necessary content into the revised version. Based on the current version and the author's response, I will raise my score accordingly.

W1

Thank you for your valuable comment regarding the architectural novelty of our work. UniTraj indeed leverages established components such as RoPE, masked modeling, and the encoder-decoder framework. But our contribution goes significantly beyond their direct application. We have customized and innovatively extended these components to address the unique challenges of trajectory data. Specifically:

• Adaptive Trajectory Sampling: We introduce the adaptive sampling mechanism to dynamically handle the varying density and irregular intervals typical in trajectory data. This allows the model to better represent dense urban traces and sparse intercity movements, enhancing its generalization and robustness.
• Trajectory-Aware Masked Modeling: Unlike conventional masked modeling in NLP or vision, we designed four dedicated masking strategies specifically for spatio-temporal sequences with irregular sampling, frequent missing points, and noise, ensuring effective learning from real-world GPS traces.
• Domain-Specific Encoding: We adapted RoPE to encode both spatial and temporal relationships jointly, rather than treating positions as a single sequence, thereby capturing the continuous and multi-dimensional nature of trajectory data.
• Unified and Multi-Granularity Pretraining: UniTraj brings together several trajectory-specific pretext tasks (e.g., next-location prediction, trajectory completion) under a single framework, enabling the model to learn diverse mobility patterns across different spatial and temporal scales.

In summary, we construct UniTraj based on proven deep learning modules, primarily driven by extensive experimental validation and customized modifications based on trajectory data. The unified design represents a potential direction for trajectory data modeling.

W2

Thank you for highlighting the limitation regarding contextual features and the focus on motorized trajectory data. We appreciate this important perspective and would like to clarify our design choices:

• Generality: Our primary objective was to develop a region-independent and modality-agnostic trajectory foundation model that can learn from raw GPS traces with minimal reliance on external data sources. While incorporating such contextual features can improve performance in localized settings, they limit transferability and are not consistently available across regions, particularly in developing countries or less-mapped areas.
• Model Robustness to Diverse Settings: Our self-supervised masking strategies and adaptive resampling help the model handle sparse, irregular, and low-speed trajectories effectively. Therefore, despite the focus on motorized data in WorldTrace, UniTraj demonstrates strong performance on multimodal datasets, such as GeoLife and Grab-Posisi, which include walking, biking, and mixed transport modes.
• Contextual Feature Integration: We agree that incorporating contextual information such as POIs and road networks can further enhance model performance. Although our current work does not exploit these features, UniTraj is modular by design and readily extended to integrate such signals as additional inputs or embedding channels in future work.

W3

Thank you for pointing out concerns about hardware requirements and reproducibility. We would like to clarify that our use of A100 GPUs was primarily for accessing the training and evaluation process on large-scale datasets, and is not a necessity for running UniTraj. Inference and fine-tuning can be performed efficiently on widely available consumer hardware, such as an NVIDIA 2080Ti. Specifically, on 2080Ti, UniTraj holds 321.68 M FLOPs. Inference time: 0.131 seconds and Memory usage: 4,241 MB for 1000 samples.

Most importantly, UniTraj is a lightweight architecture (2.38M parameters) that is significantly smaller and more efficient than prior baselines (e.g., TrajBERT and TrajFM), allowing smooth operation on non-server-grade GPUs and even for edge-level deployment with further compression. Below is our comparison of three representative models.

Q1

Thank you for your insightful observation regarding the impact of data quality and preprocessing. We agree that if sufficient prior information, such as mode labels or regularized sampling intervals, were available, data cleaning and resampling could further improve model performance.

This aligns directly with the motivation behind our ATR strategy, which is designed to handle irregular, noisy, and multimodal sampling patterns without requiring external priors. By randomly constructing a large number of possible trajectory behaviors, we aim to evaluate UniTraj's robustness and transferability under various dynamic constraints.

In our manuscript, we provide a preliminary corollary to support this perspective. Specifically, if we can convert the existing trajectory dataset to a distribution that better matches the target city, the intrinsic uncertainty (entropy) of the modeling task is reduced, making the learning problem better posed and easier for any model, like UniTraj.

Limitation

Thank you for raising the important issue of privacy in trajectory data. We fully agree that privacy is a fundamental concern in mobility data research. While our experiments utilize public and well-processed datasets such as OSM-based traces, which are less likely to contain personally identifiable information. We also recognize that privacy risks may be relevant.

As you pointed out, we discussed this issue in Section E of the appendix. In response to your suggestion, we will move and expand this discussion into the main text of the revised manuscript. Apart from general privacy risks related to trajectory data, we will also emphasize the handling of public and anonymous datasets in this study, as well as how to ensure privacy removal.

Thank you again for your suggestion, which will help us make the manuscript more comprehensive and responsible.

W1

Thank you for your comment regarding computational efficiency and inference latency. While the main submission prioritized modeling and generalization, we now provide evidence that UniTraj offers superior computational efficiency over representative trajectory models:

The above result was conducted on a server with Nvidia GeForce RTX 2080 Ti GPU and Intel(R) Xeon(R) Silver CPU. We can gain the following insights:

• Parameter Efficiency: UniTraj requires significantly fewer parameters than both TrajBERT and TrajFM, reducing memory and storage demands during deployment.
• Inference Speed: All three models achieved very fast inference speeds, with less than 0.14 seconds per 1,000 samples.
• Memory Usage: UniTraj consumes less memory than both TrajBERT and TrajFM under identical batch and sequence settings.
• Lower Computational Cost: UniTraj has the lowest FLOPs, indicating better computational efficiency.

These results demonstrate that UniTraj achieves superior computational efficiency and faster inference with a much smaller model footprint, making it better suited for real-world deployment scenarios, especially those with hardware or latency constraints.

W2

We appreciate the reviewer’s important observation regarding ethical and privacy considerations. While the main paper focused on technical contributions due to space constraints, we also carefully considered the ethical and privacy aspects of trajectory data usage in Appendix A.4. Below are key clarifications:

1. Source and Anonymity: All GPS trajectories used in our work were collected from OpenStreetMap’s public GPS trace platform, which operates under the Open Data Commons Open Database License (ODbL). These traces are user-contributed and anonymized, containing no personal identifiers, and are limited to public movement paths (e.g., roads, highways).
2. No Individual-Level Modeling: UniTraj does not infer or use any user identity or metadata. It models aggregate spatio-temporal patterns, not individual behavior, making re-identification infeasible.
3. Model Application Scenarios The model is trained and evaluated exclusively on transportation-scale movements, not on private or personal location histories. Its intended applications are in domains such as urban mobility analysis, logistics, and transportation planning, not surveillance or individualized tracking.

We will include a dedicated description in the revised manuscript to explicitly discuss these ethical considerations and privacy safeguards. Thank you again for bringing this critical aspect to our attention.

Q1

Thank you for highlighting the importance of computational efficiency at scale. We have benchmarked UniTraj and recent baselines (TrajBERT, TrajFM) on large-scale datasets, reporting both FLOPs and runtime metrics. The experiment was set up on a computing server with an Nvidia GeForce RTX 2080 Ti GPU and Intel(R) Xeon(R) Silver CPU.

Specifically, we can see that the three models have very similar inference times and can quickly complete the processing of thousands of samples. In addition, UniTraj requires substantially fewer FLOPs than both baselines, reflecting its computational efficiency. Therefore, UniTraj offers superior efficiency and scalability for both training and inference, making it well-suited for deployment on large-scale trajectory data.

Q2

Thank you for raising the crucial issue of user privacy and identifiers. Below are the safeguards we incorporated or the dataset held:

• Data Source and Licensing: All trajectory data used in UniTraj comes from OpenStreetMap (OSM) GPS traces, which are public, anonymized, and shared under the ODbL. The original data contains no user identifiers, device IDs, or metadata that can be used to trajectory individuals.

• Preprocessing for De-Identification: During dataset construction (see Appendix A), we applied filtering and spatial-temporal normalization, including short trips, loops, and localized trajectories. In addition, we also use map-matching to align raw points to road networks, further abstracting fine-grained individual movement.

• Sample Released: The publicly shared dataset includes only a processed, curated subset of trajectories.

• Model-Level Safeguards: UniTraj learns from aggregated movement patterns, not individual users, which cannot reconstruct or trace back individual identities or raw trajectories. Moreover, we do not publish fine-tuned models on private or proprietary datasets, avoiding potential overfitting to any local patterns.

W1

Thank you for raising the important point regarding potential data bias. We agree that crowd-sourced data can exhibit geographic and modality bias. To address this concern, we provide two complementary pieces of evidence that demonstrate UniTraj’s generalization ability beyond those biases:

1. We explicitly evaluated UniTraj on GeoLife and Grab-Posisi, which include mixed transportation modes beyond the WorldTrace dataset, including walking, cycling, and public transit. As shown in Table 2 of the main results section, UniTraj achieves competitive performance on these datasets. It indicates that the model effectively generalizes to non-motorized movement patterns and non-developed regions, even without being trained on those exact distributions.

2. To further investigate the potential geographic bias, we analyzed UniTraj’s performance across different continents in the WorldTrace dataset. The results are summarized below:

   	North America	Asia	Europe	Africa	Oceania	South America
   MAE	10.58	9.42	10.03	10.61	10.12	9.98
   RMSE	13.84	12.38	13.29	13.46	13.17	12.89
   Percentage	67.97	24.68	4.40	0.20	1.52	1.23

These results show that no region suffers from severe degradation, and in fact, performance in Asia and South America is comparable or better than that in North America or Europe. This suggests that UniTraj does not overfit to overrepresented regions, and that its self-supervised, region-agnostic design (e.g., no POIs, no road networks) allows for stable cross-continental generalization.

W2

We appreciate the reviewer highlighting this limitation. We agree that explicitly modeling inter-agent interactions will improve the performance of the trajectory model. We would like to clarify that the core motivation for making this decision is as follows:

• Our current design is motivated by the need to establish a strong, generalizable, and region-agnostic framework. We hope it can be usable across domains and cities where inter-agent annotations are unavailable or infeasible to collect at scale.
• Modeling multi-agent interactions typically requires high-resolution temporal alignment, shared agent IDs, and consistent scene-level context, which are often missing in large-scale, real-world GPS datasets.
• Despite modeling trajectories individually, UniTraj can achieve remarkable performance. This suggests that our self-supervised trajectory learning framework captures meaningful dynamics even without explicit interaction modeling.
• UniTraj is designed with a modular architecture: while it focuses on per-agent trajectory encoding, inter-agent relations could be integrated in downstream pipelines. This allows for plug-and-play compatibility with interaction-aware components without retraining the backbone.

W3

We thank the reviewer for this observation and valuable comment. We agree that the core components of UniTraj are built upon established foundations. However, we would like to clarify that the methodological novelty of UniTraj as outlined below:

• Domain-Specific Innovations: We designed a unique pre-training strategy based on the unique properties of trajectories. ATR is a novel combination of multi-scale and interval-consistent resampling, tailored to address the heterogeneity of real-world GPS trajectories, including variable sampling intervals, missing points, and inconsistent granularity. We adapted RoPE to encode both spatial and temporal relationships jointly, rather than treating positions as a single sequence, thereby capturing the continuous and multi-dimensional nature of trajectory data.

• Unified Pretraining: We also introduce four complementary masking strategies (STM), which are uniquely adapted to trajectory semantics. This is not a direct reuse of BERT-style masking but a task-aware design. UniTraj brings together several trajectory-specific pretext tasks (e.g., next-location prediction, trajectory completion) under a single framework, enabling the model to learn diverse mobility patterns across different spatial and temporal scales.

• Empirical Impact: Despite being an incremental architecture, our approach substantially advances in trajectory modeling, as evidenced by superior performance on diverse and challenging benchmarks.

We are committed to proving that, with the right pretraining strategy and data design, even a standard encoder-decoder architecture can achieve powerful generalization and reusability across multiple domains.

Q1

Thank you for this insightful question regarding our architectural choices.

1. Why Not Encoder-Only (e.g., BERT)?

We selected an encoder-decoder architecture to better support a broad range of trajectory modeling tasks within a single, unified framework. The encoder-decoder structure, with its “destruction and reconstruction” paradigm, allows the model to more effectively learn complex spatio-temporal dependencies in trajectory data. This design is also more flexible and plug-and-play, making it adaptable to diverse downstream tasks. Additionally, the decoder can act as a projector, further enhancing the model’s versatility for various applications.

2. Why Not Causal Masking?

For many trajectory-related tasks, especially classification or masked point recovery, utilizing both past and future context is highly beneficial. Standard causal masking restricts the model to accessing only past information, which limits its ability to recover missing or irregularly sampled trajectory points. By adopting a flexible masking strategy rather than strict causal masking, our model can seamlessly handle both prediction and imputation tasks, thus improving its overall utility and efficiency.

Q2

Thank you for your interest in the dataset. We will open-source the worldtrace dataset and ensure that its release fully complies with all relevant copyright and privacy regulations. Data sharing will follow appropriate protocols to protect individual privacy and respect data usage agreements.

Q3, Q6

We thank the reviewer for this important question. To evaluate performance across regions with different sample densities, we conducted a continent-level analysis using trajectories from each region. The results are summarized below:

From the table, we can observe that despite large differences in data volume, UniTraj maintains stable error metrics across all regions, with no drastic degradation in low-sample settings. These results demonstrate that UniTraj maintains robust generalization capabilities, even in regions with limited training data. We attribute this to the model’s region-agnostic design and pretraining objectives that effectively support generalization to underrepresented geographies, even when region-specific data is sparse.

Q4

We thank the reviewer for raising this practical concern. We conducted a comparison of inference latency, memory footprint, and FLOPs among UniTraj and two representative baselines (TrajBERT and TrajFM). The experiment was set up on a computing server with an Nvidia GeForce RTX 2080 Ti GPU and Intel(R) Xeon(R) Silver CPU. The results are as follows:

From the above analysis, we can see that the three models have very similar inference times and can quickly complete the processing of thousands of samples. However, Unitraj requires approximately half the FLOPs of TrajBERT and TrajFM, and its memory usage is also significantly reduced. These results demonstrate that UniTraj is not only more efficient in terms of memory and computation but also provides faster inference, making it well-suited for large-scale and real-time trajectory applications.

Q5

We thank the reviewer for this insightful question. To assess sensitivity to trajectory length, we grouped test samples by their length and computed the MAE and RMSE for each group:

From the table, we can observe that uniTraj demonstrates stable performance across all trajectory length ranges, with optimal results on medium-length trajectories (60–180). There is only a slight decrease in performance for very short (<60) and very long (>180) sequences. For very short trajectories, the limited amount of information makes it challenging for the model to fully capture spatio-temporal dependencies. For very long trajectories, the increased sequence length introduces more variability and noise, making the modeling task more difficult. It is worth noting that for long sequences (>180), the MAE increases by only about 3.5% compared to the overall average. This indicates that UniTraj maintains robust generalization even when handling trajectories with longer durations and higher complexity. In summary, UniTraj achieves consistently strong and stable performance across a wide range of trajectory lengths.