OSM+: Cloud-native Open Street Map Data System for City-wide Experiments

1. Better data organization could be useful, but the paper lacks novelty, and does not fit ICLR.
2. The paper mentions the sensors and the sparsity challenge. However, the paper does not provide information on the source of the sensor data, and it is not reflected in the three tables of the databases discussed. Further, the paper lacks a sufficient explanation of how the Sparsity Challenge relates to the advantages of the database .
3. Figure 6 in the paper does not seem to be referenced or explained.
4. In the experiment of traffic prediction, the OSM database uses the UTD19 dataset and is compared with the PEMS dataset. But it appears that the advantages of the OSM database are not evident from this comparison. The variance problem probably comes more from UTD19 than from the OSM database.

1. The paper uses varying terms like system, framework, and engine to describe the proposed work, which makes it confusing for me to identify differences between them.

2. The frequent use of the term "efficient" throughout the text lacks evidence (i.e., experiments) to support it.

3. Figure 4 is not referenced in the main text.

4. Some logical flow issues arise, such as in Section 4.1: "However, in recent years, it has been noticed that newly proposed methods can hardly exhibit significant improvements over existing ones. Thus, we are here to propose 31 new city-level datasets." Introducing new datasets does not necessarily address marginal improvements in existing methods. Additionally, the main motivation for proposing a large number of datasets seems to be to highlight their dynamic and sparse characteristics. However, there is no statistical evidence demonstrating how sparsity varies between cities compared to classic datasets like PEMS and METR.

5. The spatial and temporal spans of the 31 traffic flow datasets are not clearly outlined, nor is the process for data collection (e.g., traffic flow data) explained.

6. The traffic policy control experiment lacks clarity, and no results for this experiment are presented in the main text.

7. Overall, the experiments section offers limited insights that might benefit future research, aside from the city boundary detection task. The findings in the city boundary detection task seem only loosely related to the system’s contributions. It's unclear how this system differs from existing ones in identifying correlations between road networks and socio-economic indicators.

8. Each contribution mentioned by the authors is not sufficiently addressed:

• Data-wise: How is this data different from typical OSM data? If efficiency is a key feature of the proposed system, efficiency experiments should be included in the main experiments.
• System-wise: The system’s uniqueness compared to other systems is not clearly articulated.
• Application-wise: The definition and scope of applications, such as traffic signal control, are not clearly explained.
• Benchmark-wise: The motivation behind the benchmark and the relevant dataset statistics are missing.

1. The technical contribution of the paper is unclear. From the current description of the graph database and query APIs, the advancements introduced by OSM+ appear limited. Although some improvements are mentioned in Section 3, what specific technical methods does OSM+ propose to achieve these advancements?

2. The first use case on traffic prediction is well-explained; however, more comparative results with other large-scale traffic datasets, such as [1], should be provided to better demonstrate the value of OSM+ for traffic prediction tasks.

[1] Liu, Xu, et al. "Largest: A Benchmark Dataset for Large-Scale Traffic Forecasting." Advances in Neural Information Processing Systems 36 (2024).

3. The results for the other two use cases are overly simplistic. More detailed methods and comprehensive results are needed to demonstrate the effectiveness of OSM+ in these applications. Additionally, it is unclear how OSM+ contributes to research in these two areas. For instance, could the authors provide results that demonstrate the types of analyses or experiments made possible by OSM+?

1. Although the paper mentions a series of data processing steps, the lack of a detailed description of the data cleaning and transformation process puts the credibility of the data quality into question, and furthermore, the data for the paper appears to be directly imported and then converted to its corresponding data directly through osm2pqsql. https://planet.openstreetmap.org/ .
2. Although OSM+ provides standardized global road network data, this is based on existing OpenStreetMap data and fails to clearly articulate its distinctive and innovative contribution
3. While this paper cites applications such as traffic forecasting, transportation policy control, and urban boundary detection, the examples are not sufficiently in-depth to adequately demonstrate the specific ways in which OSM+ significantly outperforms other methods in these applications.
4. This paper does not provide quantitative comparisons to demonstrate the performance benefits of OSM+ over other existing tools and formats.