Temporal Logic-Based Multi-Vehicle Backdoor Attacks against Offline RL Agents in End-to-end Autonomous Driving

We thank the reviewer for the questions and will incorporate all suggested changes in our next version.

Q1: Offline RL in AD. Offline RL has been a core methodology in AD research, with studies demonstrating its performance [1], improving planning via expert knowledge [2], and deploying it in high-speed scenarios [3]. Dedicated benchmarks [4] have been introduced to evaluate offline RL driving agents. Safety-focused works [5, 6] study its reliability. Those works highlight that offline RL for AD is widely recognized and standardized.

[1] Fang et al., Offline RL for AD with Real World Driving Data, ITSC'22.

[2] Li et al., Boosting Offline RL for AD with Hierarchical Latent Skills, ICRA'24.

[3] Pan et al., Agile AD using End-to-End Deep Imitation Learning, RSS'18.

[4] Lee et al., AD4RL: AD Benchmarks for Offline RL with Value-based Dataset, ICRA'24.

[5] Lin et al., Safety-Aware Causal Representation for Trustworthy Offline RL in AD, RA-L'24.

[6] Shi et al., Offline RL for AD with Safety and Exploration Enhancement, ML4AD NeurIPS'21.

Q2: RL backdoor literature. Due to space limits, we focus on data poisoning backdoors in DRL for AD, not general RL backdoor attacks. Below is a brief summary. Prior work explored backdoor attacks in single-agent DRL using state perturbation patches as triggers [1]. [2] introduces a two-agent setting, where the adversarial agent’s actions serve as the trigger. More recent work extends backdoors to multi-agent RL [3].

[1] Kiourti et al., Trojdrl: Trojan attacks on DRL agents, DAC'20.

[2] Wang et al., Backdoorl: Backdoor attack against competitive RL, IJCAI'21.

[3] Chen et al., Marnet: Backdoor attacks against cooperative multi-agent RL, TDSC'22.

Q3: Use of third-party trajectory data, support of cited works for threat model. Using third-party trajectory data to train driving agents is a common practice. [1] trains offline RL driving agents on Waymo’s trajectory data. [2] builds an imitation learner using human trajectories. [3] uses large-scale taxi trajectories to train a driving agent. We further clarify that our threat model does not directly assume training a driving agent on third-party trajectory data. The attacker is the data annotation service. Whether the AD developers use in-house or third-party data, they outsource the annotation step. Leveraging third-party annotation services is standard across AD. NuScenes[4] and PandaSet[5] use Scale AI for labeling trajectory data, BDD100K[6] uses Amazon Mechanical Turk. [7, 8] list annotation vendors used by AD companies. Thus, our threat model is fully aligned with practice. Regarding the cited works, while [22] and [60] are not directly AD-focused, [60] evaluates their attack on AD tasks, and [22] cites AD as a key motivation. [25] supports its threat model by noting AD developers’ use of third-party annotation services.

[1] Rowe et al., CtRL-Sim: Reactive and Controllable Driving Agents with Offline RL, CoRL24.

[2] Chen et al., Learning by cheating, CoRL'19.

[3] https://wayve.ai/thinking/emerging-behaviour-of-our-driving-intelligence-with-end-to-end-deep-learning.

[4] Caesar et al., nuScenes: A multimodal dataset for AD, CVPR'20.

[5] Xiao et al., PandaSet: Advanced Sensor Suite Dataset for AD, ITSC'21.

[6] BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning, CVPR'20.

[7] Liu et al., A Survey on AD Datasets: Statistics, Annotation Quality, and a Future Outlook, IV'24.

[8] M14 Intelligence. 2020. Autonomous Vehicle Data Annotation Market Analysis.

Q4: Attacker’s knowledge of the MDP. In our setting, the attacker need not know the victim’s MDP in order to perform poisoning or target action manipulation. We directly modify different timesteps’ positions of the victim car within the training data, i.e., the attacker works entirely at the trajectory level and does not require the agent’s transition dynamics, reward, or action. For the trigger generation process, we leverage the TL framework to evaluate and keep those trajectories that satisfy our TL specifications. For target action generation, we directly perturb these coordinates to encode the target actions. Our pipeline operates at the coordinate level, so the adversary does not need to know the MDP.

Q5: Timing of poisoning. We clarify that the poisoning does not run during the entire training process. We refine the configurable behavior models of surrounding attack vehicles until the generated trajectories satisfy our TL specifications. Then we deploy these finalized behavior models with the TL-guided parameter to collect the poisoning data. Finally, we inject the poisoning data into the full training set at a specific poisoning rate, without updating it.

Q6: Attacker as provider. We agree with the reviewer that an attacker with access to both trajectory data and the MDP could typically be the agent provider. In this work, we follow existing studies and assume the attacker only needs access to the training data; thus we model them as third-party annotators. We agree that a provider‐level attack subsumes our scenario and will explicitly discuss in our revision that the adversaries can be the annotation vendor or the agent provider, depending on their available knowledge.

Q7: State/action clarification. Our state space is a 259-dimensional vector containing sensor inputs and ego-car status. Action consists of low-level control commands: steering and throttle. We describe the state, action space in Appendix B, and will move key elements to Sec. 3 in our next version.

Q8: Lowering the poisoning rate (PR). We select BadRL for its design focus on minimizing the PR. It uses a pre-trained Q-network to identify states where the target action is most harmful, and adds a mutual information regularizer. While our threat model assumes the attacker has no control over training, we relax this assumption to adapt BadRL and customize it for behavior cloning, which lacks a value function. Instead of using Q-value, we estimate the attack value using the log-probability gap between the optimal and target actions. We poison the top-k% highest-gap states, and modify the action to the target. We then introduce the mutual-information regularizer that maximizes the similarity between parameter-space gradients produced by triggered and clean versions of the same state. Tab. 1 shows that BadRL can help reduce 4% of the poisoning rate. We will include more analysis in the revision.

Tab. 1 Applying BadRL on BC (medium task).

Q9: Experimental structure. In Sec. 4.2, we demonstrate our attack's effectiveness across 3 RL algorithms, various trigger patterns, and target actions, with detailed analysis on the robustness of RL algorithms and trigger patterns’ influence on attack effectiveness. We will follow the reviewer's suggestions to improve the flow.

Q10: OOD trajectory defense. In Sec. 4.3, we evaluated our attack using a defense aimed at smoothing out OOD trajectories. We extend the defense with our ADE metrics: trajectories with ADE above a threshold (empirically set to 50) are removed. Tab. 2 shows that this defense mitigates the backdoor behavior, although it still underperforms a fully benign policy. This is likely because a single ADE threshold cannot filter out all poisoning trajectories, and more intelligent threshold selection is needed. We observe a drop in benign reward, possibly due to the removal of high-ADE trajectories that include complex traffic scenarios or failure cases, which improves the agent’s robustness. We believe this defense is promising, and we leave it as our future work.

Tab. 2 Performance comparison under ADE defense.

Q11: Domain-specific insights & pseudo triggers (PT). Our main contribution is not the trajectory trigger itself, but the introduction of complex, customizable multi-vehicle triggers. The proposed TL-driven procedure can automatically generate those trigger trajectories, which are not discussed in existing works. [5]'s PT differs from our setting as [5] presents a defense method where the PT is the detection agent’s trajectory that causes another agent to fail. While we act as the attacker, crafting stealthy backdoors that avoid unintended activations. For visual similarity, [5] notes that PT can diverge from the original trigger. Our “falsely trigger trajectories” are very similar to the true triggers. This precision guarantees that the target action is only activated under the specified trigger pattern.

Q12: Negative-training (NT) trade-offs. To evaluate the trade-offs, we prepare three policies: a backdoored policy with and without NT, and a clean policy. We simulated realistic perception noise by adding zero-mean Gaussian noise to the ego car’s LiDAR points, whose default detection radius is 50m. We defined low, medium, and high noise levels with std of 5,10, and 15cm, and applied noise at every time step. In Tab. 3, the backdoor remains effective under all noise levels, denoted by the relatively lower poisoned reward. However, NT increases the policy’s sensitivity to noise, as the benign and poisoned rewards are all lower compared to the policy without NT.

Tab. 3 NT trade-offs.

We thank the reviewer for the questions and will incorporate all suggested changes in our next version.

Q1: The reviewer raised the concern about the diversity of trajectory-based triggers for the backdoor attack.

We thank the reviewer for raising this concern. Our temporal logic specification framework is inherently flexible and supports the definition of diverse trigger patterns. To address the reviewer’s concern, we design three additional trigger patterns: 1) one vehicle bypasses from one side while another suddenly brakes, 2) four vehicles bypass the ego car—two doing so synchronously, followed by another two synchronously, and 3) an asynchronous bypass, where two vehicles bypass the ego car with a 10-second interval. We evaluate these patterns on easy tasks using the Coptidice-trained agents, with a poisoning rate of 10%. As shown in in Tab. 1 below, our attack can support a diverse range of trigger designs. While all triggers are effective, the attack performance is slightly reduced for more complex patterns, such as the four-vehicle bypass. This is likely due to the increased state complexity introduced by multiple vehicles, which may require a higher poisoning rate to be fully effective. We will incorporate the results, analysis, and videos in our next version.

Tab. 1 Attack effectiveness of three new trigger patterns.

Additionally, we present the results of single-vehicle trigger patterns in Tab. 11 in Appendix D.4, further demonstrating the expressiveness of our framework. That said, there is a trade-off between trigger complexity and stealthiness: overly complex triggers may be difficult for the model to learn and for the attacker to successfully inject, while overly simple triggers may occur frequently in everyday scenarios, making them more detectable.

Q2: The reviewer asked what would happen if no other vehicles are present in the environment, and whether a pedestrian or bicyclist could instead serve as the trigger.

We thank the reviewer for the question. We would like to clarify that our threat model assumes the attacker can deliberately and strategically control one or multiple vehicles and drive along specific, physically plausible trigger trajectories; these motions will be observed by the ego car’s LiDAR and activate the backdoor. The attacker’s vehicles respect safety rules and do not collide with the ego car. As a result, our trigger does not rely on ambient or “natural” traffic flow, and the attacker controls the interacting vehicles as needed. In the hypothetical case where no other vehicles are present and the attacker also does not introduce one, the backdoor cannot be activated, which is consistent with our model. Furthermore, to address concerns about accidental activation, we conduct experiments in Appendix D.1 to demonstrate that our designed trigger patterns are rare during a daily life driving scenario, indicating a low likelihood of false positives in natural traffic.

Our poisoning does not depend on the type of triggers. To use bicyclists or pedestrians as triggers, we replace the default vehicle behavior model with one that captures their specific dynamics. MetaDrive already includes built-in dynamics models for bicyclists and pedestrians, so we simply swap in those models when generating trigger trajectories for these object types. For bicyclists, the simulator approximates the rider and bike as a single rigid‐body box whose motion is updated kinematically: at each simulation step, the engine assigns a desired linear velocity to the body, allowing it to follow any prescribed 2-D trajectory while still participating in collision checks of the simulator. Pedestrians are treated identically, differing only in their collision envelope shape and dimensions. In our experiments, we use the “sync-bypass” as the trigger path, swapping out the vehicle as a bicyclist and a pedestrian, respectively, and training the agent with Coptidice. Results in Tab. 2 show that for bicyclists and pedestrians, the backdoor still triggers but causes slightly less significant reward loss compared to the vehicle. This is likely because bicyclists and pedestrians generate weaker occlusion and kinematic disruption than the vehicle. The benign driving is barely affected, denoted by the similar benign reward, ADE, and MVR, compared to the car-based trigger.

Tab. 2 Attack effectiveness of “sync-bypass” trigger when replacing vehicle with bicyclist and pedestrian.

Q3: The reviewer pointed out the consistency in the format of section heads.

We thank the reviewer for pointing this out and will modify it in our next version.

We thank the reviewer for the questions and will incorporate all suggested changes in our next version.

Q1: The reviewer raised the concern that the experiments are mainly conducted in a simulator, with no real-world evaluation, also suggested evaluating the robustness against different weather conditions.

We thank the reviewer for the insightful comment. We agree with the reviewer that testing on real-world vehicles will demonstrate the practical effectiveness. We do not perform real-world evaluations mainly because executing backdoor attacks’ target action in the real world, such as sudden left turns, raises serious safety concerns and could result in dangerous situations. Additionally, to the best of our knowledge, no prior work on backdoor attacks in AD has been demonstrated on real vehicles. We follow existing work setups [1, 2, 3] and conduct our experiments in the well-established simulator with closed-loop evaluation. Furthermore, deploying end-to-end offline RL in a production vehicle requires extensive engineering and still remains an active research challenge; thus, it is beyond this paper’s scope. Lastly, our simulator adheres to realistic physical constraints and traffic rules. For example, when vehicles are off the road, the simulator automatically terminates the trajectory. Thus, we believe that our simulator-based results sufficiently establish the attack’s feasibility and impact, and we leave real-vehicle testing to future work.

To assess robustness under varying weather, we simulate fog and rain in the simulator. For fog, we impair LiDAR accuracy by randomly dropping 5% of the LiDAR points from the ego car’s state vector and setting those points to 0. For rain, we reduce tire-road friction by lowering the wheel friction parameter in the ego car’s dynamics model from the default value of 0.9 to 0.5. We evaluate in the medium tasks using BC and Coptidice trained agents, the target action is “sudden left turn” and the trigger pattern is “sync-bypass”, poisoning rate as 10%, following the same setup as Tab. 2 in the paper. The results are presented in Tab. 1 below. First, we observe that the noise will degrade the clean model’s performance. For backdoored policies, the injected trigger still remains effective, as evidenced by a clear performance gap between the benign and poisoned metrics. This demonstrates that our attack can still remain effective under different weather conditions, although the benign metrics are influenced due to the presence of the noise. All results are averaged over 100 trajectories. We will include these results in our next version.

Tab. 1 Attack effectiveness under two weather conditions on the medium task.

[1] Gong et al., Baffle: Hiding backdoors in offline reinforcement learning datasets, S&P'24.

[2] Pourkeshavarz et al., Adversarial Backdoor Attack by Naturalistic Data Poisoning on Trajectory Prediction in Autonomous Driving, CVPR'24.

[3] Han et al., Physical backdoor attacks to lane detection systems in autonomous driving, ACM ICM'22.

Q2: The reviewer raised the concern about the practicability of our threat model, having access to the whole training data can be challenging.

We thank the reviewer for the question. Our threat model assumes that the attacker is the third-party data annotation service, an entity to which the AD developers outsource the labeling process of the trajectory data. This setup grants the attacker full and reasonable access to the training data.

Leveraging third-party annotation services (e.g., Scale AI, Amazon Mechanical Turk, etc.) is a well-established and widely adopted practice in the AD field. For example, leading datasets such as nuScenes [1] and PandaSet [2] explicitly credit Scale AI for labeling trajectory data, while BDD100K [3] leverages Amazon Mechanical Turk. Both academic surveys [4] and empirical industry studies [5] document an entire ecosystem of external annotation vendors that AD companies rely on. Thus our assumption that the attacker is from the third-party annotation service with access to the whole training data is consistent with real-world practice. Furthermore, we assume the attacker has access only to the data, not the training algorithm, which is fully aligned with our defined threat model about the data annotation service.

[1] Caesar et al., nuScenes: A multimodal dataset for autonomous driving, CVPR'20.

[2] Xiao et al., PandaSet: Advanced Sensor Suite Dataset for Autonomous Driving, ITSC'21.

[3] BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning, CVPR'20.

[4] Liu et al., A Survey on Autonomous Driving Datasets: Statistics, Annotation Quality, and a Future Outlook, Intelligent Vehicles 2024.

[5] M14 Intelligence. 2020. Autonomous Vehicle Data Annotation Market Analysis, https://m14intelligence.com/public/index.php/product/11.

Q3: The reviewer suggested considering more diverse behavior of attacker vehicles, e.g., incorporating traffic rules into the temporal logic specifications.

We thank the reviewer for the insightful suggestion. Following the reviewer’s suggestion, we consider two traffic rules, one is speed limits, another is lane keeping. Specifically, alongside the existing atomic proposition that captures vehicle position, we introduce two new atomic propositions for speed and lane membership: (1) $\mu_{\text{speed}}^{i}(t,v_{\max}) \:\Leftrightarrow\ v_i(t)\le v_{\max}$ (2) $\mu_{\text{lane}}^{i}(t,\mathcal{L}) \:\Leftrightarrow\ (x_i(t),y_i(t))\in\mathcal{L}$. where $\mathcal{L}$ is the polygonal region of the current lane. We add them to our existing temporal logic predicates and define two new traffic-rule constraints directly within our propositions.

$`SpeedLimit`(v_{\max},[t_s,t_e]) := \mathbf{G}{[t_s,t_e]}\,\mu_{\text{speed}}^{i}(t,v_{\max})$

$`LaneKeep`(\mathcal{L},[t_s,t_e]) := \mathbf{G}{[t_s,t_e]}\,\mu_{\text{lane}}^{i}(t,\mathcal{L})$

We use Coptidice and extend the traffic rules on our sync-bypass behavior, meaning that when the attacker performs this, it strictly follows the speed limits or lane keeping. The target action is to brake suddenly. We set the speed limit as 60mph. Results from Tab. 2 below show that incorporating traffic rules into the attacker's trajectory still yields effective attack performance, demonstrating that our framework is flexible and supports the definition of common traffic rules into attacker behavior. All results are averaged over 100 trajectories.

Tab. 2 Attack effectiveness of two new trigger patterns incorporating traffic rule compliance.