Uncertainty-aware Human Mobility Modeling and Anomaly Detection

Thanks for your comments, we address your concerns from the following aspects:

(1) Experiment and explanation of how $\lambda$ is tuned/selected in the loss function: In our paper, we manually decide $\lambda$, so that the loss of numerical features and discrete features can be in the same scale.

(2) More baselines: We appreciate your recommendation to strengthen the baseline methods. In designing our baselines, we aimed to cover core models that serve as the foundational backbone of many state-of-the-art approaches in human activity sequence modeling. For example, we included Transformer as a baseline, which has been foundational to SOTA human activity/trajectory sequence methods like [1-4]. Our goal was to ensure that the baselines represent essential frameworks widely recognized for their effectiveness in this domain, thus providing a fair comparison to our approach. Nevertheless, we value your suggestion and will consider incorporating additional SOTA baselines to enrich the evaluation further.

[1] Lin, Yan, et al. "Pre-training context and time aware location embeddings from spatial-temporal trajectories for user next location prediction.", AAAI, 2021.

[2] Chen, Yile, et al. "Robust road network representation learning: When traffic patterns meet traveling semantics." CIKM, 2021.

[3] Jiang, Jiawei, et al. "Self-supervised trajectory representation learning with temporal regularities and travel semantics." ICDE, 2023.

[4] Yang, Sean Bin, et al. "Lightpath: Lightweight and scalable path representation learning.", KDD 2023.

(3) A section to introduce anomaly detection in sequential data (e.g., trajectory and time series): Thanks for the question. Trajectory anomaly detection methods cannot be used to solve our event ad task directly. Their settings are different. Trajectories anomaly detection aims to make the decision given the original and destination, while in our task, we do not specify a given OD pair.

(4) Theoretical complexity analysis. Given a sequence of $L$ events, with each event associated with $F$ features. The time complexity in the intra-event attention layer is $F^2 \cdot d$, and the complexity in the inter-event attention layer is $L^2 \cdot d$. Let $T$ denote the sample times at the inference process, the total complexity of the proposed model is $T(F^2 \cdot d + L^2 \cdot d)$. In comparison, the complexity of the vanilla transformer is $L^2 \cdot d$. In contrast to the vanilla transformer, our proposed model sacrifices the complexity slightly but enhances the model's capacity to detect feature correlations within a single event and introduces the capability to learn uncertainties. This trade-off is deemed appropriate as our focus is on offline anomaly detection, where real-time decision-making is not required.

We hope that we have fully addressed your concerns, and we look forward to your further input and your support of the paper on the confidence score.

We hope that we have addressed your concerns, and we are looking forward to your further input and your support of the paper in score.

(4) Concerns on Experiment Setup. We'd like to address your concerns from the following points: Firstly, we did not have access to the original simulation model itself and we could only access the simulated dataset it produced. This means we were unable to use the generative model directly for anomaly detection. Furthermore, the purpose of the simulation model was solely to generate synthetic data for research purposes, rather than for anomaly detection tasks. Secondly, the generation model was learned on real samples from 2017 National Household Travel Survey (NHTS) [1] and anomalies are specifically designed and injected to reflect realistic abnormal behaviors in human mobility. This setup aims to capture a wide range of patterns, including long-tail distributions, to imitate real-world scenarios as closely as possible. Thirdly, there is no publicly available real human mobility dataset with known anomaly labels. Even for available human mobility datasets, manually labeling anomalies would be impractical due to the scale and complexity of the data.

[1] Stanford, Chris, et al. "NUMOSIM: A Synthetic Mobility Dataset with Anomaly Detection Benchmarks." Proceedings of the 1st ACM SIGSPATIAL International Workshop on Geospatial Anomaly Detection. 2024.

(5) [R2Q5]Details on model design. Thanks for the comments. The design of Eq.7 is mainly inspired by [1]. In the paper, we present Eq.7 as "average predicted logits, normalized over all possible categorical outcomes", which utilizes softmax and therefore introduces the exponential term. The reason for +1 in eq.10 is to let the anomaly score be more numerical stable in case $\alpha_f + \beta_f$ equals 0.

Concern regarding multiple passes for the categorical loss but not for the numerical features. The AU of the numerical feature is modeled as a learnable loss attenuation term, which can be learned in a one-forward pass. However, to model the AU for categorical features, it is not trivial to use the same loss function as in numerical features. Following [1], for the categorical feature, we model the AU term $\sigma_f$ as the standard deviation of the logits (see Eq.6), which requires multiple passes to learn the AU term, so that we can penalize less when the data has high uncertainty.

[1] Alex Kendall and Yarin Gal. 2017. What uncertainties do we need in Bayesian deep learning for computer vision?

Thanks for such detailed and constructive comments! we address your concerns in the following points:

(1) Broader impact of our work. Thanks for raising this question. We would like to argue that our paper does not only target at transportation audience. Indeed, our method can be easily generated to sequence modeling tasks in many other domains. We kindly directly you to the "Broader Impact" section in the appendix: "This approach is highly generalizable and can be applied across various domains, such as finance (e.g., detecting credit card fraud or money laundering), online shopping (e.g., analyzing user purchasing patterns and identifying bot activity), and app user behavior analysis (e.g., detecting shifts or unusual engagement)."

(2) Motivation of abnormal mobility behavior detection. The motivation for detecting abnormal mobility behaviors lies in the potential to address critical societal needs in public health, safety, and urban management. For example, during a pandemic, detecting anomalies helps monitor social distancing and identify potential hotspots, aiding health officials in preventing the spread of infectious diseases [1,2]. Similarly, unusual movement patterns can signal emergencies like natural disasters and extreme weathers, allowing for faster, more targeted responses that enhance urban resilience [3]. Beyond these, abnormal mobility detection supports crowd management and public safety by identifying patterns that may indicate overcrowding or potential security threats [4]. In urban planning, detecting these anomalies allows city planners to better understand evolving trends in transportation which can help optimize infrastructure and improve public transit services [5]. Each of these applications demonstrates the importance of human mobility anomaly detection.

[1] Shearston, Jenni, et al. "Social-distancing fatigue: Evidence from real-time crowd-sourced traffic data", Sci Total Environ, 2021.

[2] Nakamoto, Daisuke, et al. "The impact of declaring the state of emergency on human mobility during COVID-19 pandemic in Japan", Clin Epidemiol Glob Health, 2022.

[3] Li, Xiang, et al. "Using human mobility data to detect evacuation patterns in hurricane Ian", Annals of GIS, 2024.

[4] Sanchez, Thomas, et al. "The Implications of Human Mobility and Accessibility for Transportation and Livable Cities", Urban Science, 2023.

[5] Shuo, Shang, et al. "Human Mobility Prediction and Unobstructed Route Planning in Public Transport Networks", MDM, 2014.

(3) Concerns on baselines. We appreciate your recommendation to strengthen the baseline methods. In designing our baselines, we aimed to cover core models that serve as the foundational backbone of many state-of-the-art approaches in human activity sequence modeling. For example, we included Transformer as a baseline, which has been foundational to SOTA human activity/trajectory sequence methods like [1-4]. Our goal was to ensure that the baselines represent essential frameworks widely recognized for their effectiveness in this domain, thus providing a fair comparison to our approach. Nevertheless, we value your suggestion and will consider incorporating additional SOTA baselines to enrich the evaluation further.

[1] Lin, Yan, et al. "Pre-training context and time aware location embeddings from spatial-temporal trajectories for user next location prediction.", AAAI, 2021.

[2] Chen, Yile, et al. "Robust road network representation learning: When traffic patterns meet traveling semantics." CIKM, 2021.

[3] Jiang, Jiawei, et al. "Self-supervised trajectory representation learning with temporal regularities and travel semantics." ICDE, 2023.

[4] Yang, Sean Bin, et al. "Lightpath: Lightweight and scalable path representation learning.", KDD 2023.

Thanks for your comments, we address your concerns from the following aspects:

(1) Concerns on novelty. Thanks for your comments. Our main contribution is not the uncertainty modeling technique itself; in fact, we contribute by being the first to model both the aleatoric and epistemic uncertainty in behavior sequence learning, which can be easily generated to other domains with both numerical and categorical features. We appreciate your recommendation to explore alternative methods. These approaches are indeed promising, and we leave it as future work to further enhance our uncertainty estimation.

(2) Details on the experimental setting of baseline methods For SS-MLP, we searched embedding dimensions of 512 and 1024 with architectures of 7, 9, and 10 layers. We experimented with two input feature sets: one with latitude, longitude, and start time, and another adding stay duration and POI. We pick the best result from the experiments to report in the comparison. For STOD and RioBus, we used embedding sizes of 512 and 1024. For LSTM and Transformer settings, please refer to Appendix Section C, where we detail the parameter search space shared across all deep models. The best-performing configuration is reported in the comparison.

(3) Experiment on real-world datasets. Thanks for the advice. There is no publicly available real human mobility dataset with known anomaly labels. Even for real human mobility data, it would be infeasible to manually inspect and label anomalies due to the complexity and scale of the data.

(4) Other anomaly scoring methods. Our anomaly scoring method computes the deviation between the predicted and true values for each event feature, which is similar to reconstruction error, as it reflects the difference between input features and their predicted (or reconstructed) values. We have already experimented with three different scoring methods in Section 4.4.

We hope that we have addressed your concerns, and we are looking forward to your further input and your support of the paper in score. Thank you.

Thanks for the contrastive comments, we address your concern in the following points:

(1) Concerns on novelty in Uncertainty Quantification. Thanks for your comments. Our main contribution is not the uncertainty modeling technique itself, in fact, we are the first to model both the aleatoric and epistemic uncertainty in human behavior sequence learning. MC Dropout is only a component of our work, which equips the model with epistemic uncertainty learning ability.

(2) Details about SimLA. The SimLA dataset is a private dataset, and there are currently no plans for public release. It is generated using the same simulation process as the publicly available NUMOSIM dataset.

(3) Reason of choosing MC Dropout for EU. We chose Monte Carlo Dropout for epistemic uncertainty estimation because (1) it is a simple yet effective approach that has been extensively validated across various domains; (2) it can be seamlessly incorporated with aleatoric uncertainty modeling. (3) it is a practical choice for our initial experiments while maintaining strong empirical performance. We appreciate your recommendation to explore alternative methods, such as deep ensembles and evidential neural networks. These approaches are indeed promising, and we leave it as future work to further enhance our uncertainty estimation.

(4) Downstream tasks. There are multiple downstream tasks that the proposed model can be helpful.

1. Pandemic Monitoring and Public Health: During a pandemic, detecting mobility anomalies can help track compliance with social distancing and identify areas with unexpected gatherings, which helps health officials enforce measures and allocate resources. [1]

2. Urban Resilience to Emergencies: Anomalous movement patterns, such as sudden evacuations, can signal emergencies like natural disasters or extreme weather. Detection supports efficient response by directing resources to affected areas [2][3].

3. Urban Safety and Crowd Management: Identifying deviations in movement patterns can help detect public safety risks, including crowd congestion, unexpected gatherings, and potential terrorism. [4]

4. Transportation Planning: Anomalies in mobility patterns reveal issues like road closures or transit delays, enabling city planners to optimize routes, reduce congestion, and improve transit services. [5]

[1] Shearston, Jenni, et al. "Social-distancing fatigue: Evidence from real-time crowd-sourced traffic data", Sci Total Environ, 2021.

[2] Nakamoto, Daisuke, et al. "The impact of declaring the state of emergency on human mobility during COVID-19 pandemic in Japan", Clin Epidemiol Glob Health, 2022.

[3] Li, Xiang, et al. "Using human mobility data to detect evacuation patterns in hurricane Ian", Annals of GIS, 2024.

[4] Sanchez, Thomas, et al. "The Implications of Human Mobility and Accessibility for Transportation and Livable Cities", Urban Science, 2023.

[5] Shuo, Shang, et al. "Human Mobility Prediction and Unobstructed Route Planning in Public Transport Networks", MDM, 2014.