XXLTraffic: Expanding and Extremely Long Traffic forecasting beyond test adaptation

We appreciate the reviewer's acknowledgment of the dataset’s impact and the careful review for our manuscript.

Q1: Road infrastructure may influence the long time span dataset.

The dataset is valuable, but performing extremely long-term predictions is challenging, as traffic patterns can change significantly over the years, and even the road infrastructure itself may be reshaped over a 23-year span. Such factors could potentially reduce the impact of the dataset's contribution, because many open datasets are enough for normal short and long term prediction.

We appreciate your feedback and understand your concern regarding the factor of traffic patterns changing over a 23-year span. Therefore, we set the gap years to several years, with the gap length referencing the time it takes for a highway to be planned and finally completed. Predicting the traffic volume at those locations several years in advance is of great significance. As you can see from Table 1 in the paper, the performance of all baselines is relatively poor, indicating the considerable difficulty of accurately predicting this kind of setting. Thus, our dataset spanning up to 23 years can provide more samples for training and offer more potential for modeling temporal features in the models.

Q2: Some typos.

Some typos, e.g., ‘beyongd’ should be ‘beyond’ around line 130;

Thank you for your feedback. We will carefully review the entire manuscript to correct any grammatical errors and typos. These are some error corrections from our self-review as follows (the full table could be seen in the author global rebuttal):

(1) In Section 1.1, "Our dataset are evolving and longer than existing datasets." should be "Our dataset is...".

(2) In Section 4.1, "tfNSW is an open-source data platform provided by Transport for NSW" should be "The tfNSW is...." because "tfNSW" is the first word of the sentence.

(3) In Section. 5.4, "SOTA" should be "the state of the art" since it is being used for the first time.

Additionally, Based on Reviewer ytp5’s suggestion, we have added five additional baselines shown in the table:

Based on Reviewer 6qSU’s suggestion, we have provided statistics on training times for the largest and smallest datasets shown in the following table, demonstrating that our dataset and the settings we propose can be easily trained on a standard single V100 GPU.

We sincerely thank the reviewer for taking the time to provide valuable suggestions and detailed comments. We are pleasant to hear that the reviewer'acknowledges of importance of introducing large-scale datasets with extremely long time spans.

Q1: Lack of model contribution

The most significant weakness is that this paper has no novelty. Although this conference has a dataset and benchmarks area, I do not think, for a top-tier conference, it is enough to propose a large-scale dataset without any model contribution. Given the fact that there are existing large-scale datasets available with just the number of nodes or time spans fewer than XXTraffic, my concern becomes more intense.

We appreciate your feedback and understand your concern regarding the novelty of our submission. However, we would like to point out that many esteemed dataset contributions in the field have focused primarily on the datasets themselves and the associated baseline experiments, without necessarily introducing a new model. Examples include the Large-ST dataset from NIPS 2023 and the DL-Traffic dataset, which was a runner-up for the best paper at CIKM 2021. Neither of these proposed their own models, yet they are recognized for their significant contributions to the research community.

We firmly believe that the data processing, experimental setup, and the extensive baseline comparisons we have presented hold practical research value. In response to the concerns raised by Reviewer ytp5 and Reviewer 6qSU, we have added five additional baselines and included statistics on training times for the largest and smallest datasets, demonstrating that our dataset and the settings we propose can be easily trained on a standard single V100 GPU.

We maintain that our work provides a substantial contribution to the field by offering a unique dataset that enables long-term traffic forecasting with gaps, a challenging domain that has not been sufficiently explored. The practical implications of our dataset and experimental framework are significant, and we are confident that they will facilitate further research and innovation in traffic forecasting.

[1] Liu X, Xia Y, Liang Y, et al. Largest: A benchmark dataset for large-scale traffic forecasting[J]. Advances in Neural Information Processing Systems, 2024, 36.

[2] Jiang R, Yin D, Wang Z, et al. Dl-traff: Survey and benchmark of deep learning models for urban traffic prediction[C]//Proceedings of the 30th ACM international conference on information & knowledge management. 2021: 4515-4525.

Q2: Unclear destription of some motivation

Some motivations remain unclear. In the introduction section, the authors mentioned ‘beyond test adaptation’. However, it is not easy to understand how this relates to the proposed datasets. Is the concept of ‘beyond test adaptation’ better or more realistic than test time adaptation?

Thank you for your suggestion. We have clarified the concepts of 'beyond test adaptation' and 'test time adaptation' in the introduction section to distinguish our approach from test time adaptation. Our aim is to provide a dataset with an extended time span that can accommodate various durations of gap settings, such as 1 year, 5 years, and in this work, we have recommended 1, 1.5, and 2 years for baseline comparison experiments. Under different gap settings, we can consider them as different distributions. We will expand Figure 1 in subsequent versions to include gap settings and clarify the concept more clearly.

We sincerely thank for the constructive comments and suggestions. We are encouraged that the reviewer acknowledges that our work provide a good guide for peers to understand the latest technological developments.

Q1: Spatial size of the dataset

The maximum number of nodes in the proposed dataset is less than the Large-ST, which seems to need improvement, after all, the real road network is very large.

Thank you for your review. We appreciate the feedback provided. In the hyperlink in our manuscript, we have made available the raw dataset which contains a substantial number of nodes. However, our focus is on conducting extremely long-term forecasting (i.e., our gap setting, such as gaps of 1 year, 1.5 years, and 2 years) based on nodes that span over twenty years. However, most of these nodes do not have data spanning over 20 years, which is essential for our gap setting. After conducting data analysis and preprocessing, we have established that each district contains between 100 and 800 nodes. This is why our spatial node count does not reach the substantial numbers seen in the Large-ST dataset.

Despite the relatively lower number of nodes per district in our dataset, we offer datasets from various geographical areas, including 9 districts from the PeMS dataset and 1 traffic dataset from the state of New South Wales, Australia. This diverse regional coverage allows for a comprehensive exploration of extremely long-term traffic forecasting across different spatial scales.

Q2: explain in more depth the motivation for dataset

I think it is necessary for the authors to explain in more depth the motivation for constructing such a long-term dataset, because as transportation infrastructure advances and human travel modes change, traffic data that is too ancient is not always helpful enough to understand future transportation patterns.

Our motivation for creating a extremely long-term dataset is to provide a robust basis for future traffic pattern analysis, which is essential for strategic planning in urban development and infrastructure investment. Our dataset, combined with our gap setting, allows for the assessment of traffic volumes over extended periods, enabling better planning for future road construction and traffic management. This extremely long-term perspective is crucial for aligning transportation infrastructure with future demands and ensuring the sustainability of urban environments.

Q3: computational resources for the baselines experiments

**Due to the sheer size of the proposed dataset, it requires more computational resources. Therefore, whether this dataset can be popularized and whether peers can easily adopt it in their own research is also a concern. **

Refer to your Question 1, our dataset is designed to filter out nodes with a time span exceeding 20 years, resulting in a relatively small spatial dimension that meets the criteria. Consequently, the dataset primarily has a longer temporal dimension, making the training cost acceptable. We have compiled the training time for one epoch of all the baselines on the largest dataset, PEMS04_gap, and the smallest dataset, tfNSW, as shown in the table below. (Additionally, we have included five traffic forecasting domain baselines: STGCN, ASTGCN, GWN, AGCRN, PDFormer according to Reviewer ytp5’s suggestion, as shown in Table 1 in the global comments.)

Table 2. Training time per epoch pf all the baselines in largest PEMS04_gap and smallest tfNSW

Q3: Additional traffic forecasting baselines

Therefore, based on the five baselines you provided, We have observed that most traffic forecasting baselines indeed perform worse compared to those in long-term forecasting, and their performance deteriorates as the gap increases, as shown in the following table (We also provided a traing time statistics icluding the aforementioned baselines and your suggested baselines in Table 2 of global rebuttal):

We thank the reviewer for taking the time to assess our manuscript and offering valuable suggestions.

Q1: Insufficient contribution

Compared to existing works like LargeST, this work's contribution is insufficient as it merely expands the temporal and spatial scope of data collection. These data can be obtained through the open-source PEMS system in the same way.

We acknowledge your concern regarding the comparison with existing works like LargeST and the perceived insufficiency in our contribution. It is true that our dataset, like LargeST, is derived from the open-source PEMS system. However, our primary contribution lies in the unprecedented expansion of the temporal dimension, which is a significant departure from datasets that focus more on spatial expansion, such as LargeST. Our dataset's unique value proposition is its ability to support extremely long-term traffic forecasting with gaps, a capability not commonly found in other datasets. This feature is particularly relevant for practical applications such as long-range urban planning and infrastructure development, which require traffic volume assessments years in advance. By providing data that spans over two decades, we enable researchers and planners to assess traffic patterns and trends over an extended period, which is crucial for strategic planning and decision-making. Furthermore, we have enriched our dataset with various baselines to highlight the challenges of the traffic forecasting domain, especially with our gap setting. This approach not only demonstrates the dataset's utility but also contributes to the field by providing a benchmark for evaluating the performance of different models in long-term forecasting scenarios. In summary, while our dataset may not exceed LargeST in terms of spatial dimensions, its extensive temporal coverage and the inclusion of gap settings offer a novel and valuable resource for traffic forecasting research and applications.

Q2: Regional limitation of the dataset

Although the number of regions and time span are substantial, the vast majority of data is limited to California. Including data from more cities and countries might have been a better approach.

Yes, we agree with your perspective that focusing solely on traffic data from California is limiting. Datasets from different countries and regions would indeed enrich the understanding of traffic conditions in various cities. Therefore, our dataset also includes a 13-year traffic dataset from New South Wales, Australia, which is complemented by the same set of baselines to ensure comparability and analysis across different geographical contexts.

Q3: Additional traffic forecasting baselines

Most baselines in the experiments are specifically designed for time series prediction, lacking models specifically designed for traffic prediction/spatiotemporal prediction, such as PDFormer[1], ASTGCN[2], STGCN[3], AGCRN[4], GWN[5], etc. It is important to explore the performance of these types of baselines on traffic datasets.

We agree with your suggestion. The reason we chose baselines based on the long-term forecasting domain for our experiments is that our experimental setup is more closely aligned with multivariate long-term forecasting. Since the introduction of Informer, the development of baselines in this field has been particularly vibrant, and we have benefited from the widely available open-source repositories that facilitate easy migration and reproduction. However, we acknowledge that it is also necessary to consider baselines from the traffic forecasting domain (12-step forecasting).