TopoLogic: An Interpretable Pipeline for Lane Topology Reasoning on Driving Scenes

Thanks for your careful reading of our paper. We hope our response and clarifiction can ease some of your concerns and you could reconsider your rating.

Q1: Why the paper does not compare with TopoMLP?

A1: Thank you for your insightful question. I will respond to it in detail. Although both TopoMLP and TopoLogic use ResNet-50 as their backbone, TopoMLP is implemented using PETR, while TopoLogic, TopoNet, SMERF, and LaneSegNet are based on DETR. Unfortunately, TopoMLP could not be run on the NVIDIA RTX 3090 GPU due to its high memory consumption. As a result, we were unable to perform a comparative analysis with TopoMLP at that time due to equipment limitations. We apologize for this inconvenience. To address this issue, we have implemented TopoLogic based on PETR and conducted experiments on Tesla A100 GPU. The experimental results are provided below, and we will include these results in the final version of the paper.

The experiments were conducted on subset_A, OpenLane-V2 version 2.1.0. In this context, TopoLogic* denotes the version of TopoLogic implemented using PETR.

Q2: TopoMLP outperforms TopoLogic on TOP$_{lt}$?

A2: As indicated in the table in A1, TopoMLP does not outperform TopoLogic on TOP$\_{lt}$ (25.4 v.s. 26.8). We implemented TopoLogic using PETR, and it shows improved performance on TOP$\_{lt}$ (26.8 v.s. 25.4 v.s. 23.4).

Q3: Not fully sure if this is a Neurips paper, since technical contributions are marginal.

A3: Thanks. The performance can significantly improved (+11.4 TOP$\_{ll}$ in Table 5) even if it is only used as post-processing, and can be further improved (+13.0 TOP$\_{ll}$ in Table 1) after integrated training. Such an improvement exceeds the improvement of previous works in a large margin, which in itself is a valuable contribution to the field. reasonable analysis and effective verification are our academic contributions, which will be beneficial to promote the development of the field.

Q4: Fair comparison about mapping function.

A4: Thank you for your insightful feedback. We appreciate your suggestion to ensure a fair comparison by considering a hand-tuned decision boundary. In response to your concern, we conducted additional experiments to further validate the performance of $f_{ours}$ Gaussian function with its adaptive decision boundary.

To determine the decision boundary, we explored two hyperparameters: $\alpha$ and $\lambda$. We evaluated the TOP$\_{ll}$ metrics across various combinations of these hyperparameters. The results are presented in the following table:

As the table illustrates, the decision boundary with $\alpha=1.3$ and $\lambda=0.23$ yielded the best performance in terms of the TOP$\_{ll}$ metric. This finding aligns with our original experimental results and underscores the optimization effectiveness and advantages of the $f_{ours}$ Gaussian function. We hope these additional results address your concerns regarding fair comparison and demonstrate the robustness of our approach.

Q5: $f_{tan}$ in equation 6 seems to map to [0, 2] and not to [0, 1].

A5: Thank you for your detailed consideration. In fact, since $f$ maps endpoint distances, so $x$ is always greater than or equal to 0 rather than unbounded, ensuring that the range of these functions is always within [0, 1].

Q6: Equation 10 is not a [0, 1] probability, and do you perform some sort of normalization?

A6: Thank you for your attention. We did not perform normalization. The two coefficients, λ₁ and λ₂, in Equation 10 are learnable and can be trained to range between [0,1]. In other words, λ₁ + λ₂ approaches 1.

Q7: If you say a variable is 1x1 do you mean it is a real number? If so just write \in R.

A7: Yes, it's not necessary to say 1x1 shape for scalars, and we're going to modify the expression.

Q8: The batch size of 2 on 8 GPUs seems strange, is it a batch size of 2 per GPU, so an actual batch size of 16?

A8: Your understanding is correct. To be precise, the batchsize of each GPU is 2, so the total batchsize of 8 GPUs is 16. It's our fault to make it unclear.

Q9: How do you handle traffic elements?

A9: For the traffic element treatment, we adopt the same strategy as baseline method TopoNet, i.e., predict 2D bounding box of traffic elements using DETR, and reason topology between them with lane utilizing vanilla MLP. Since our work focused on lane topology, we rarely mentioned traffic elements, which will be added in the final version.

Q10: Typos.

A10: Thank you for carefully pointing out these typos. We will diligently correct these issues.

We hope this response could help address your concerns, and wish to receive your further feedback soon.

Thanks for your careful reading of our paper. We hope our response and clarifiction can ease some of your concerns and you could reconsider your rating.

Q1: Lane segment is not discussed in the methodology session.

A1: Thanks for your attention to this detail, which is indeed omitted from the Methods section. However, lane segment is just another more refined representation of the lane, and the centerline can still be easily extracted from the lane segment, so that our algorithm does not need any special processing for the lane segment.

Q2: It is not stated which split of the dataset is used for experiments.

A2: Thanks for your mind. Perhaps we should this information in the caption of tables, even although these information can be found on line 233-234 and 183-185 (SOTA comparison, Table 1 on subset_A/B & Table 2 on subset_A) and line 248-250 (ablation study, Table 3-5 on subset_A).

Q3: The mixture of using different versions of the metric is confusing (vx.0 and vx.1).

A3: It is indeed unusual to using two versions of metric, but it is a legacy issue in the lane topology field, as can be seen in the Openlane-V2 official repository (Update on OpenLane-V2 Metric · Issue #76 · OpenDriveLab/OpenLane-V2 · GitHub). We provide two versions of the metric for comparison with more methods. Moreover, it should be noted that our method has a significant advantage over each other regardless of which version of metric.

Q4: Why detection scores of the proposed pipeline also improved compared to other methods?

A4: The reason for this can be found in the caption of Figure 2 and Introduction (Line 61-62), and we will add further explanation in the experiment section. Note that the proposed modules serve in each layer of the decoder, where geometric distance topology and lane similarity topology are fused into the final lane topology. This lane topology is facilitated to augment lane learning by GNN aggregating features from adjacent lanes in the next decoder layer. Thus, Improvement on TOP$\_{ll}$ corresponds to increased DET$_l$（28.6 to 29.9 (+1.3) in Table 1).

Q5: How these two modules benefit existing networks?

A5: Thank you for your valuable suggestions. We carried out experiments by freezing the network parameters of TopoNet and incorporating the similarity and geometric distance modules for end-to-end training. The results in the table indicate that the two modules introduced in TopoLogic significantly enhance TopoNet. Additionally, training TopoLogic end-to-end yields better performance compared to freezing a portion of the parameters.

Q6: Why improving the similarity between two connected lanes would improve topology reasoning performance?

A6: Thank you for your valuable question; it is very meaningful. The similarity module proposed in TopoLogic calculates the similarity of lane endpoints rather than the entire lane itself. As shown in Figure 1, the lane query passes through two independent MLPs to encode representations, computing the similarity between the starting and ending points of two directed lanes. Additionally, to address your inquiry, we conducted experiments with MLPs as shown in the table. The "No MLP" configuration does not encode the lane query, calculating the similarity of the entire lane. In contrast, the "Single MLP" configuration computes the similarity of the same endpoint across two lanes. The table indicates that employing two independent MLPs to encode lane queries for calculating the similarity between the starting and ending points of directed lanes yields the best performance, thereby validating the effectiveness of the TopoLogic design.

We hope this response could help address your concerns, and wish to receive your further feedback soon.

Thank you very much for your quick response.

Q7: For Q2, are the scores reported on the validation set or test set?

A7: We're sorry we didn't mention that detail, and we will add more clarification in the final version. In fact, all the results are reported on the validation set, which is completely consistent with the protocol of the all reference paper, e.g., TopoNet, SMERF, LanSegNet, etc. As for test set, it is only used for the closed-evaluation of the OpenLane-V2's challenge(https://opendrivelab.com/challenge2024/#mapless_driving), which was not available at the time of our research.

Q8: For Q5, the DET$_t$ of the frozen TopoNet drops compared to the baseline version, are there modules which could affect the traffic element detection branch?

A8: We're sorry to confuse you. Either similarity module or geometric distance module just involves the lane, not traffic elements, so DET$_t$ really shouldn't drop. However, in the freezing experiment we loaded the weights of the TopoNet that we reproduced, whose DET$_t$ in itself is lower (46.9 v.s. 48.6) than reported in the TopoNet paper. It seems to be a bug in baseline's official code. The reproduction about DET$_t$ was always unstable and lower than the value reported in the original paper, and this phenomenon can also be observed in the results of other paper (e.g., 44.5 DET$_t$ of TopoNet in Table 1 in paper SMERF$^{[1]}$)

[1] Luo K Z, Weng X, Wang Y, et al. Augmenting lane perception and topology understanding with standard definition navigation maps[J]. arXiv preprint arXiv:2311.04079, 2023.

If you have any further questions or suggestions about our article, we'd love to discuss our content with you.

Thanks for your careful reading of our paper. We hope our response and clarifiction can ease some of your concerns and you could reconsider your rating.

Q1: Like an incremental work based on TopoNet. The contribution is limited. The contributed part is only a similarity module for mapping lane line endpoints. But the design is simple and straightforward, not surprising enough.

A1: Thanks for your comments. Compared to TopoNet, our modification looks like small, but that doesn't mean the contribution is small.

• We clearly point out that TopoNet and other methods ignore the endpoint-connection characteristics between the lanes when topological reasoning, and design a module based geometric distance and a module based on semantic similarity to significantly improve the topological performance (+13.0 TOP$\_{ll}$ in Table 1) in the case of reducing the number of parameters and the amount of computation. Therefore, the contributed part is not limited in similarity module.
• As similarity module, its subtlety lies in that it uses two independent MLPs to map the lane query rather than a single MLP, which can decouple a lane into two queries of start and end point, achieving an analogous effect to the (endpoint) geometric distance module in semantical space. This opinion can be verified by the following experiments, so it is not simple and straightforward.

Q2: The proposed GeoDist module is a post-processing algorithm. It's like an engineering improvement rather than academic contribution.

A2: Thanks for your comments, but we don't think it's just an engineering improvement.

• The GeoDist module is not just used as post-processing after training. This module can be used (in conjunction with Similarity module) at each layer of the decoder, so that the lane topology is enhanced geometrically and semantically at each layer and refined layer by layer. In addition, lane topology in layer is also facilitated by GNN to augment lane learning in layer through aggregating features from adjacent lanes. Using GeoDist module in an integrated training manner provides more performance gains (+13.0 TOP$\_{ll}$ in Table 1) than just post-processing (+11.4 TOP$\_{ll}$ in Table 5).

• This improvement is not an accident attempt, but comes from our accurate and highly interpretable understanding of the endpoint-connection characteristics of the lane topology, which makes it apart from simple engineering improvements.

  All in one, reasonable analysis and effective verification are our academic contributions.

Q3: When compared with previous methods (like TopoNet, SMERF), what about the computation overhead and the difference in network architecture?

A3: Thank you for your valuable question. The key distinction between TopoLogic and both TopoNet and SMERF is that TopoLogic introduces similarity module and geometric distance module for lane topology reasoning, whereas TopoNet and SMERF only use MLP for lane topology reasoning. Here, we provide some computational benchmark as follows, and we will include the experiment result in the final version of the paper.

We hope this response could help address your concerns, and wish to receive your further feedback soon.

Thanks for your careful reading of our paper. We hope our response and clarifiction can ease some of your concerns.

Q1: The proposed method is relatively simple and involves only post-processing modules, which at the same time leads to limited contributions and incremental improvements.

A1: Thanks for your comments.

• The proposed method, TopoLogic, is an end-to-end trained model with two key modules (i.e., Similarity & GeoDist) instead of just one simple post-processing. The modules are plugged at each layer of the decoder, so that the lane topology is enhanced geometrically and semantically at each layer and refined layer by layer. Once trained, the GeoDist module can be utilized as a plug-and-play post-processing component for any pre-trained lane topology reasoning model to enhance TOP$\_{ll}$.

• Although the model is relatively simple, the performance improvement is significant (10.9 to 23.9 (+13.0) on TOP$\_{ll}$ in Table 1) after end-to-end training. The improvement in itself is a valuable contribution to the field, and its theoretical basis (endpoint-connection characteristics of the lane topology) is another contribution to the field.

Q2.1: Considering that TopoNet is just a simple baseline model, the boost in topological reasoning is not surprising.

A2.1: Thank you for your feedback. We are to blame for the ambiguity, and we will add more clarification in the final version. TopoNet is the official recommended baseline evaluated on OpenLane-V2 benchmark, and recently many new model (e.g., SMERF, LaneSegNet) are actually based on TopoNet, so we claim vaguely that our TopoLogic is based on TopoNet. We conducted experiments with TopoNet, SMERF, and LaneSegNet. As demonstrated in Table 1 and Table 2, TopoLogic consistently achieves superior performance compared to these approaches.

Q2.2: The improvement of is very small and even decreases, which needs further analysis and discussion.

A2.2: Thanks for your comments, and we will add more clarification in the final version.

• It is reasonable that the improvement on TOP$_{lt}$ is smaller the improvement on TOP$\_{ll}$, as our method mainly affects the lane-lane topology and does not deal with traffic elements specifically.

• For the same reason, DET$_t$ has not improved. As for decrease sometimes, it seems to be a bug in baseline's official code. When we reproduced the DET$_t$ of baseline, it was always unstable and lower than the value reported in the original paper. We also found that the DET$_t$ 's reproduced result in other papers (e.g. SMERF) were lower.

Q3.1: It may be better to manually set the hyperparameters for a series of gradients to observe its effect on the final performance.

A3.1: Thank you for your feedback. We tested a series of manually set $\alpha$ and $\lambda$ values to observe their effects on performance. As is shown in following table, automatically learned hyperparameters $\alpha=1.3, \beta=0.23$ are actually superior to other manually set value, which frees us from the hassle of tuning parameters.

Q3.2: Why $𝑓_{𝑔𝑎𝑢}$,$𝑓_{𝑠𝑖𝑔}$,$𝑓_{𝑡𝑎𝑛}$ are very close but differ dramatically on TOP$_{𝑙𝑙}$ in Table 3?

A3.2: Thank you for your thoughtful question. We believe the counterintuitive phenomenon is primarily due to the sensitivity of these functions to the threshold values. To investigate this hypothesis, we conducted additional analyses by plotting Gaussian function curves with varying $\alpha$ and $\lambda$ parameters, as illustrated in Figure 1 and 2 of the Rebuttal PDF. These figures and table in A3.1 show that while the threshold values do not exhibit significant changes, the TOP$\_{ll}$ scores vary substantially, which is another reason why we adopt learnable mapping functions in TopoLogic.

Q4.1: The motivation of Lane Similarity Topology is to alleviate the influence of inaccuracies in lane detection on topology reasoning. However, the proposed method as a post-processing step does not significantly change the lane line detection results.

A4.1: Nice question. We are to blame for the ambiguity, and we will add more clarification in the final version. Our motivation is indeed to alleviate the influence of inaccuracies lane detection on topological reasoning, however, our strategy is not to directly improve lane detection, but to robustly improve lane topological reasoning through the geometric and semantic information of the lane in the case of inaccurate detection, which is a brand new perspective first proposed by us for the topological reasoning task .

Q4.2: In addition, the comparison results of the Lane Similarity Topology module should be added to Figure 4 to increase its persuasiveness.

A4.2: Thanks for your valuable suggestions, and we will add the module's visualized results to Figure 4. Since OpenReview's reply cannot directly attach pictures, please go to Rebuttal PDF Figure 3 to view.

Q5: Typos

A5: Thank you for carefully pointing out these typos. We will diligently correct these issues.

We hope this response could help address your concerns, and wish to receive your further feedback soon.