SafeAuto: Knowledge-Enhanced Safe Autonomous Driving with Multimodal Foundation Models

Q3: The evaluation does not seem to adequately address edge cases or scenarios where traditional models might fail. A deeper examination of how the framework handles uncommon but critical driving situations could provide insights into its robustness and safety.

Thank you for your insightful suggestion. In our work, we focus on enhancing both high-level and low-level action predictions from MLLMs in autonomous driving by introducing the PDCE loss for improved numerical precision in low-level control and implementing post-safety verification using Markov Logic Networks to ensure that any unsafe actions generated by the MLLM are identified and corrected before execution. Additionally, our multimodal RAG approach leverages previous driving scenarios to reduce hallucinations and enhance prediction reliability.

While we have included an example in Figure 5 of Appendix A.1 demonstrating the rejection and correction of aggressive behavior in one critical driving scenario, we acknowledge the need for a more comprehensive examination of how our framework handles uncommon but critical driving situations. Thank you for helping us improve our work.

Q4: Can the authors elaborate on the specific mechanisms used to extract and integrate environmental predicates into the retrieval process? How do these predicates influence decision-making during high-level action predictions?

Thank you for the insightful question. To extract and integrate environmental predicates into the retrieval process, we first concatenate all environmental predicates into a single binary vector. This vector is then processed by a MLP to map it into a hidden embedding that aligns with embeddings from other modalities, such as video and control signals.

We found that incorporating these binary vectors is crucial for enhancing retrieval performance, as demonstrated by our ablation study in Table 6. The results show that the average action accuracy improves by 30% when this binary compact information is included. This significant improvement indicates that the original video embeddings contain considerable noise, and integrating environmental predicates helps achieve more accurate and reliable retrieval. By leveraging these predicates, our model can make more informed and precise high-level action predictions. We will emphasize this point more in our revision.

Q5: The paper mentions a gradual increase in the value of σ during training for the PDCE loss. How was this approach determined, and what empirical evidence supports its effectiveness compared to alternative strategies?

Thank you for the insightful question! Initially, we used a fixed value of σ for training. Although this approach already outperformed the baseline that employs pure Cross-Entropy loss, we observed that the model struggled to learn the target distribution effectively at the beginning (since the $\sigma$ is a bit large). To address this, we experimented with gradually (exponentially) increasing σ from 0.01 to 0.35 during training, transitioning from an easier-to-learn distribution to the target distribution. This gradual increase resulted in a more stable training process and improved performance, as demonstrated by our empirical results below:

The results indicate that the gradual increase in $\sigma$ leads to further reductions in RMSE for both speed and course predictions compared to the fixed $\sigma$ approach. Therefore, we adopted this strategy as it proved to enhance model performance more effectively. Thank you for pointing out this, and we will add this statistic into our revision for better clarification!

Q6: How does the framework handle potential conflicts between high-level predictions and low-level control signals?

Thank you for your valuable question. Currently, we do observe some potential conflicts between high-level predictions and low-level control signals, although these cases are relatively rare. At present, we mitigate these conflicts implicitly by introducing more relevant context through retrieval from the multimodal RAG, which helps the LLM reduce potential conflicts. Additionally, our experiments on larger internal datasets indicate that increasing the amount of training data can also gradually decrease the frequency of these conflicts. We recognize that currently we can only mitigate the issue but cannot fully solve it. Addressing these conflicts more explicitly is indeed an important direction for our future work and warrants further exploration.

Q3: The loss PDCE formulation is unclear. Any new loss function should be defined clearly with notation. Here the description of the loss is unclear. The properties of the loss needs to be studied in a more varied setting. The authors just produce a graph for a specific setting to motivate this loss, but better experiments need to be planned to demonstrate the behavior of this loss. Please Formulate the loss function in mathematical terms, and also perform experiments on loss behavior with error bars and more varied situations

Thank you for your valuable feedback regarding the formulation and clarity of our PDCE loss. We apologize for any confusion caused and appreciate the opportunity to clarify our approach comprehensively.

To address the limitations of the standard Cross-Entropy loss in handling numerical values, we introduce the PDCE loss. Below is the precise mathematical formulation, which is also detailed introduced from line 204 to line 246 in paper:

$\mathcal{L_{\text{PDCE}}} = \sum_{i=1}^{n} w_i \cdot KL(\mathcal{P}_i \parallel \mathcal{D}(\mu_i, \sigma))$

where:

• $n$ is the number of digits in the numerical string.
• $\mathcal{P}_i$ represents the predicted probability distribution for the $i^{th}$ digit.
• $\mathcal{D}(\mu_i, \sigma)$ is a digit-level discrete Gaussian distribution centered at the true digit $\mu_i$ with a standard deviation $\sigma$, this ensures that digits closer to $\mu_i$ have higher probabilities, mimicking the behavior of MSE loss in promoting numerical closeness.
• $w_i$ is the positional weight for the $i^{th}$ digit, decreasing with the digit's position (e.g., higher weights for more significant digits). These weights are derived using a Gaussian distribution to emphasize the importance of higher-order digits.

Intuition and Motivation

The standard CE loss evaluates predictions at the token (digit) level, which can lead to scenarios where numerically closer predictions do not necessarily correspond to lower loss values. For example, predicting "11.99" for the target "12.46" may incur a higher CE loss than predicting "14.46," despite "11.99" being numerically closer to "12.46." This discrepancy arises because CE loss prioritizes the correctness of individual digits over the overall numerical proximity.

By contrast, our PDCE loss incorporates a Gaussian-based weighting mechanism that emphasizes the significance of higher-order digits and allows for a degree of flexibility in lower-order digits. This alignment with MSE loss ensures that predictions closer to the target in a numerical sense result in lower loss values.

Properties of PDCE Loss

• Numerical Closeness: PDCE loss ensures that numerically closer predictions yield lower loss values when we represent them as strings, akin to MSE loss.
• Positional Weighting: Higher-order digits are given more weight, reflecting their greater impact on the numerical value.

Experimental Validation

We also have conducted extensive experiments to validate the effectiveness of the PDCE loss:

1. Ablation Studies:

   • PDCE vs. CE Loss: As shown in Table 13 in Appendix B.1, replacing the standard CE loss with PDCE resulted in a significant reduction in Root Mean Squared Error (RMSE), demonstrating improved numerical prediction accuracy.

2. Hyperparameter Analysis:

   • We explored different values of $\sigma$ in the PDCE loss (refer to Fig. 4), observing that the PDCE consistently outperformed the CE loss across various settings, further confirming its robustness.

3. Behavioral Analysis:

   • Fig. 3(b) illustrates the loss landscape post fine-tuning with PDCE, showing a bell-shaped distribution that aligns with the desired numerical proximity behavior.

In summary, the PDCE loss effectively bridges the gap between CE and MSE losses for numerical string predictions by ensuring that numerically closer predictions correspond to lower loss values. This is achieved through a combination of positional weighting and a Gaussian-based probability distribution.

We will enhance the clarity of our PDCE loss formulation and its properties in our revision. Should you have any further questions or require additional clarifications, please feel free to let us know!

Q4: The paper is in the context of knowledge-enhanced autonomous driving and argues that the underlying models are limited as they do not account for any available knowledge when making their decision. Specifically, the authors capture the knowledge rules in the form of logical formulae, yet the related work section does not discuss prior works that would be relevant here (e.g., representing and integrating background knowledge into neural network models).

Thank you for your valuable feedback! We apologize for the oversight in our related work section. Initially, our intention was to keep the discussion focused and concise, primarily centered around the discussion on the application of MLLM in autonomous driving. As a result, we inadvertently provided less emphasis on the integration of knowledge into neural network models. Realizing this might lead to confusion for some readers, we will revise the section to include more pertinent related works on this topic, below is a part of it which is related with MLN.

“Specifically, Markov Logic Networks are commonly used to represent and integrate background knowledge into neural network models. In [1], the authors first explore the possibility of encoding background knowledge, such as the shape of a stop sign for image classification, and derive the corresponding strict robustness guarantees for the predictions. Building on this foundation, [2] scales the approach by incorporating Graph Convolutional Networks to encode the logical formulas, thereby enhancing the model's ability to capture complex relational structures. Additionally, they leverage variational inference techniques to efficiently approximate the posterior distributions, facilitating scalable and robust learning.”

We will also include more related work on this topic like probabilistic circuits [3] in our revision and thank you again for your thorough review; your comment is greatly appreciated!

[1] Yang, Z., Zhao, Z., Wang, B., Zhang, J., Li, L., Pei, H., ... & Li, B. (2022). Improving certified robustness via statistical learning with logical reasoning. Advances in Neural Information Processing Systems, 35, 34859-34873.

[2] Zhang, J., Li, L., Zhang, C., & Li, B. (2023, February). CARE: Certifiably robust learning with reasoning via variational inference. In 2023 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) (pp. 554-574). IEEE.

[3] Choi, Y., Vergari, A., & Van den Broeck, G. (2020). Probabilistic circuits: A unifying framework for tractable probabilistic models. UCLA. URL: http://starai. cs. ucla. edu/papers/ProbCirc20. pdf, 6.

Q5: Improvements in clarity and readability are required. For example, in Figure 2, the term "cumulative_probs" (line 231) could be confusing as it suggests traditional cumulative probabilities, whereas "weights" might be a clearer term, aligning with the main text (line 238). Additionally, there are repetitions, such as in lines 83-84: "regarding high-level action prediction, a significant limitation of current methods in high-level action prediction." Also, in line 193: "with temperature as 1.0," it is unclear what is meant here, as the temperature concept is introduced later on line 374.

Thank you for your careful and attentive review; we truly value it! We apologize for the issues with readability. We just found the incorrect word spacing resulted from a failed compilation of the LaTeX symbol --- intended for —. This has been corrected in our revision.

Additionally, we acknowledge the potential confusion caused by the term "cumulative_probs" and have replaced it with "weights" in the revision to enhance clarity. We have also addressed the issue of repetition within the text.

Regarding the term "temperature," the usage in line 193 actually refers to the standard sampling temperature for large language models, whereas in line 374, it pertains to the distillation temperature used in learning the rankings from text embeddings. Although both instances use the term "temperature," they relate to different contexts and processes. We will clarify these distinctions more thoroughly in our revision to prevent any confusion. We greatly appreciate your detailed feedback!