Flow Matching-Based Autonomous Driving Planning with Advanced Interactive Behavior Modeling

We appreciate the constructive comments from the reviewer. In hope of addressing the reviewer's confusion, we provide the following response.

Q1: How do you ensure that each trajectory can be divided into exactly K segments with L_seg points per segment? How is this handled in cases where trajectories vary in length or have fewer points than required?

In practical implementation, we adopt fixed trajectory lengths for both training and testing, consistent with real-world vehicle deployment. On top of this, given the predefined segment length and overlap length as hyperparameters, we can divide the trajectory into desired segments with overlap accordingly (see Section 3.2 for details). If handling variable-length inputs is required, standard padding and masking techniques can be employed to ensure consistent input dimensions across all samples.

Q2: Could you elaborate on the advantages of adopting a learning-based framework in this context, despite the relatively lower performance? What are the trade-offs in terms of generalizability, scalability, or adaptability?

• Learning-based frameworks have largely replaced rule-based methods in academia [1] and industry [2] due to their scalability and data-driven generalization capabilities. However, trade-offs still exist: while data scales up, the scarcity of high-quality data leads to suboptimal performance in interactive scenarios and increasing computational demands when scaling up models. To address this, we propose Flow Planner—an efficient framework that achieves SOTA performance on limited nuPlan training data while maintaining fast inference speeds.
• We further note that the benchmark could potentially be hacked by rule-based/hybrid methods through metric optimization - they could select trajectories specifically to maximize scores [3][4]. Flow Planner could similarly employ a post-processing module for this purpose. We implemented the same post-processing module as in [5] and achieved competitive performance with existing rule-based and hybrid methods.

Q3: Could you clarify why optimization-based planning methods, such as reinforcement learning–based approaches, are not included for comparison?

As discussed in Related Works, RL has limitations such as safe exploration and reward design, which restrict its application in real world autonomous driving. In the nuPlan simulation, RL methods can leverage sufficient interactions and carefully designed rewards to achieve higher evaluation scores. However, imitation learning directly mimics the data without incorporating additional prior knowledge, making it inappropriate for direct comparison with RL approaches.

Extra Experiment Results

To further demonstrate the effectiveness of our model in interactive behavior modeling, we conducted extra experiments on interPlan [6], a closed-loop driving benchmark extended from nuPlan, which contains 335 long-tail traffic scenarios including jaywalkers (Jaywalk), accidents (Nudge Around), and dense-lane-change situations (High Traffic Density) etc, posing a huge challenge in interactive behavior modeling. We achieved SOTA performance on this benchmark.

*: prior knowledge is used for the model

In addition, we conducted ablation studies on interPlan to further demonstrate the improvement brought about by each key designs, proving the effectiveness of these modules in interactive behavior modeling.

[1] Hu, Yihan, et al. "Planning-oriented autonomous driving." Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2023.

[2] Hwang, Jyh-Jing, et al. "Emma: End-to-end multimodal model for autonomous driving." arXiv preprint arXiv:2410.23262 (2024).

[3] Cheng, Jie, Yingbing Chen, and Qifeng Chen. "Pluto: Pushing the limit of imitation learning-based planning for autonomous driving." arXiv preprint arXiv:2404.14327 (2024).

[4] Dauner, Daniel, et al. "Parting with misconceptions about learning-based vehicle motion planning." Conference on Robot Learning. PMLR, 2023.

[5] Zheng, Yinan, et al. "Diffusion-based planning for autonomous driving with flexible guidance." arXiv preprint arXiv:2501.15564 (2025).

[6] Hallgarten, Marcel, et al. "Can vehicle motion planning generalize to realistic long-tail scenarios?." 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2024.

We would like to first highlight our appreciation to the acknowledgement to our work from the reviewer. Regarding the remaining concerns of the reviewer, we provide the following response.

W1: Architectural refinements rather than paradigm shifts

We propose a coordinated solution for interactive behavior modeling in autonomous driving, with architectural refinements as a core component. Our Fine-grained Trajectory Tokenization enables kinematically continuous trajectory representation while supporting localized feature extraction. When integrated with CFG, which dynamically reweights neighboring interactions during inference, this framework establishes a new paradigm for interactive behavior generation.

W2, W3 & L1: Marginal improvement and Exclusive Reliance on nuPlan

• To our knowledge, Flow Planner is the first learning-based method to achieve a score above 90 on nuPlan without post-processing or prior knowledge. This accomplishment is particularly significant as the scores are averaged across 1,000+ val14 scenarios, making further improvements exceptionally challenging.
• To further demonstrate the effectiveness of our method, we conducted additional experiments on the interPlan [1], a benchmark tailored to better evaluate the capability and robustness of algorithms in interactive scenarios with specially modified scenarios. Our method remained SOTA on this augmented benchmark, with a significant lead over existing methods.

*: prior knowledge is used for the model

• In addition, we ablated our key designs on this benchmark to further demonstrate the effectiveness of our method in highly interactive scenarios. The proposed modules bring a total gain of +20.55, with a prominent improvement in interactive scenarios Nudge Around (+22.35), High Traffic Density (+29.03) and Jaywalk (+15.58) respectively. We also compared our Flow Planner with a vanilla imitation learning-based (Vanilla IL), which directly regresses the target trajectory, and Diffusion adaptation. Our Flow Matching-based model significantly outperforms the imitation learning (+29.43) and Diffusion adaptation (+21.11). It is worth noting that the Diffusion adaptation is trained with the same epochs of Flow Planner, which is half the steps of Diffusion Planner [2], indicating the advantage of fast convergence of Flow Matching.

• We appreciate the reviewer's suggestion to evaluate on NavSim. However, this benchmark focuses on pseudo-closed-loop testing for end-to-end driving systems, which differs from our work's scope. We would like to leave the end-to-end adaptation of our method for future study.

W3: Inadequate Safety-Critical Scenario Analysis.

We appreciate the emphasis on safety in autonomous driving. As noted earlier, we have expanded our evaluation to include safety-critical interactive scenarios. In addition, we are confused with the reviewer's discussion on Table 5 and would like to be provided with further comments from the reviewer.

[1] Hallgarten, Marcel, et al. "Can vehicle motion planning generalize to realistic long-tail scenarios?." 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2024.

[2] Zheng, Yinan, et al. "Diffusion-Based Planning for Autonomous Driving with Flexible Guidance." The Thirteenth International Conference on Learning Representations.

We really appreciate the reviewer for the constructive comments and positive feedback on our paper.

W1: The model was tested only on nuPlan dataset and no others.

• We thank the reviewer for the constructive comment and understanding. To provide a more comprehensive evaluation of the effectiveness of our method, we conducted extra experiments on interPlan [1], a closed-loop driving benchmark extended from nuPlan, which contains 335 long-tail traffic scenarios including jaywalkers (Jaywalk), accidents (Nudge Around), and dense-lane-change situations (High Traffic Density) etc, posing a huge challenge in interactive behavior modeling. We achieved SOTA performance on this benchmark over the existing baselines with a significant lead (61.82).

*: prior knowledge is used for the model

• In addition, we ablated our key designs on this benchmark to further demonstrate the effectiveness of our method in highly interactive scenarios. The proposed modules bring a total gain of +20.55, with a prominent improvement in interactive scenarios Nudge Around (+22.35), High Traffic Density (+29.03) and Jaywalk (+15.58) respectively. We also compared our Flow Planner with a vanilla imitation learning-based (Vanilla IL), which directly regresses the target trajectory, and Diffusion adaptation. Our Flow Matching-based model significantly outperforms the imitation learning (+29.43) and Diffusion adaptation (+21.11). It is worth noting that the Diffusion adaptation is trained with the same epochs of Flow Planner, which is half the steps of Diffusion Planner [2], indicating the advantage of fast convergence of Flow Matching.

W2: Clarity issues.

We appreciate the constructive suggestion from the reviewer. We will improve the clarity of paper in the final version.

Q1: How do the authors define interactive driving behavior? It is left undefined as a whole and seems quite subjective.

We appreciate the reviewer's question and apologize for any confusion caused by unclear definitions. In this work, 'interactive driving behavior' specifically refers to a vehicle's ability to perceive and respond to other road users. As previously mentioned, we conducted additional interactive scenario testing to validate the effectiveness of our design on interPlan, providing a more objective and comprehensive evaluation of our method.

Q2: What is the multi-modality of the model? Is it just in reference to unconditioned and conditioned classifier-free guidance?

• Multi-modality driving behavior describes the diverse possible actions a driver may take in a given scenario (e.g., turning left or right to avoid an obstacle). Conventional imitation learning struggles with this as it regresses to mean ground truth values. This limitation motivates our use of probabilistic generative models like Flow Matching, whose advantage is shown in the experiment results above.
• In terms of the Classifier-Free Guidance, it is adopted to amplify the awareness of the existence of neighboring vehicles in trajectory generation via combining the unconditioned and conditioned velocity field during inference. This novel implementation is proved to be beneficial in enhancing model's interactive modeling capability, as is shown in Table 3 and Table 5, as well as the extra experiments we provided above.

[1] Hallgarten, Marcel, et al. "Can vehicle motion planning generalize to realistic long-tail scenarios?." 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2024.

[2] Zheng, Yinan, et al. "Diffusion-Based Planning for Autonomous Driving with Flexible Guidance." The Thirteenth International Conference on Learning Representations.

We thank the reviewer for the constructive comments. Regarding the concerns of the reviewer, we provide the following responses with extra experiments.

To further demonstrate the effectiveness of our design, we used interPlan [1], a closed-loop driving benchmark extended from nuPlan, which contains 335 long-tail traffic scenarios including jaywalkers (Jaywalk), accidents (Nudge Around), and dense-lane-change situations (High Traffic Density) etc, posing a huge challenge in interactive behavior modeling. We remain SOTA on this benchmark, with a significant lead over previous methods.

*: prior knowledge is used for the model

In addition, we conducted further ablation studies on the interPlan benchmark, and compared our model with a vanilla imitation learning-based model Vanilla IL, which directly regress the target trajectory, and a Diffusion adaptation. The proposed modules bring significant performance improvement in interactive scenairos, and our method outperforms the imitation learning-based and diffusion adaptation with a prominent margin.

W1 & L1: The novelty of this work is limited and incremental.

Interactive behavior modeling remains a fundamental challenge in autonomous driving. Our core contribution is an integrated framework where each component is carefully designed to enhance interactive behavior modeling.

<!-- It is unfair to claim there is a lack of novelty. -->

• Fine-grained Trajectory Tokenization Our Fine-grained Trajectory Tokenization (acknowledged by reviewer yVKT & bpBq) enables kinematically continuous trajectory representation while supporting localized feature extraction, providing a solution to the overcompression problem in existing methods [2][3][4]. The Fine-grained Trajectory Tokenization enables high-quality and flexible trajectory generation (Fig. 3) and brings performance improvement in interactive scenarios, as is indicated in the table above.
• Interaction-enhanced Spatiotemporal Feature Fusion Our Interaction-enhanced Spatiotemporal Feature Fusion (acknowledged by reviewer eK33) effectively integrates heterogeneous scene information through cooperative design with trajectory tokenization, contrasting with brute-force transformer blocks[5] or unnecessarily complex architectures in prior work[6]. This mechanism prominently enhances model's robustness, with a +14.08 gain in overall performance on interPlan.
• Flow matching Flow Matching-based generative models excel at capturing multi-modal driving behavior, essential for modeling complex interactive scenarios in autonomous driving[2][3]. Their straightforward loss function leads to faster convergence, more stable training, and higher inference efficiency than diffusion models [7]. As shown in the table, our Flow Matching model outperforms both the vanilla imitation learning model (+29.43) and the Diffusion adaptation (+21.11), demonstrating the effectiveness of Flow Matching-based model in modeling interactive behavior. Notably, the Diffusion adaptation is trained for the same 200 epochs as Flow Planner—only half the steps used by Diffusion Planner [2]—highlighting Flow Matching’s training efficiency.
• Classifier-free guidance While CFG is widely used in diffusion models, our work novelly conditions on neighboring vehicles, effectively enhancing interactive behavior modeling for autonomous driving (acknowledged by reviewer bpBq & eK33). Unlike typical CFG use in tasks with explicit conditions (e.g., text prompts [8] or rewards [9]), interaction in driving lacks clear definitions. We find that neighboring vehicles provide a powerful implicit condition. By learning both unconditioned and scene-conditioned behaviors, CFG captures interaction patterns—offering a novel solution for generating interactive behaviors. As shown, our model achieves 61.82 on interPlan vs. 58.22 without CFG, with notable gains in Nudge Around (+8.50) and High Traffic Density (+7.13).

In hope of better addressing the reviewer's concerns, we welcome any further, more specific questions or comments regarding the novelty of our work.

W2 & L2: The experiments are not adequate to show the effectiveness of the paper.

• We evaluate our method on nuPlan, the largest open-source real-world driving dataset supporting closed-loop testing. As the standard benchmark for planning methods, nuPlan better aligns with real-world applications. Our comparisons already have included existing SOTA methods.
• To demonstrate the effectiveness of our designs, we further compared our method with existing competitive models on interPlan. We remain SOTA performance on this benchmark, with a significant lead (61.82) over existing models, demonstrating the effectiveness and robustness in interactive behavior modeling of our designs. The additional ablation on interPlan further indicates the effectiveness in interactive scenarios of each module we propose, as shown in the table above and discussion on W1 & L1.

W3: The layouts of table 4 and figure 3 can be improved

We appreciate the constructive suggestion from the reviewer. We will improve the layouts of the mentioned table and figure in the final version of our paper.

[1] Hallgarten, Marcel, et al. "Can vehicle motion planning generalize to realistic long-tail scenarios?." 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2024.

[2] Zheng, Yinan, et al. "Diffusion-Based Planning for Autonomous Driving with Flexible Guidance." The Thirteenth International Conference on Learning Representations.

[3] Liao, Bencheng, et al. "Diffusiondrive: Truncated diffusion model for end-to-end autonomous driving." Proceedings of the Computer Vision and Pattern Recognition Conference. 2025.

[4] Cheng, Jie, Yingbing Chen, and Qifeng Chen. "Pluto: Pushing the limit of imitation learning-based planning for autonomous driving." arXiv preprint arXiv:2404.14327 (2024).

[5] Cheng, Jie, et al. "Rethinking imitation-based planners for autonomous driving." 2024 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2024.

[6] Huang, Zhiyu, Haochen Liu, and Chen Lv. "Gameformer: Game-theoretic modeling and learning of transformer-based interactive prediction and planning for autonomous driving." Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023.

[7] Lipman, Yaron, et al. "Flow matching for generative modeling." arXiv preprint arXiv:2210.02747 (2022).

[8] Ramesh, Aditya, et al. "Hierarchical text-conditional image generation with clip latents." arXiv preprint arXiv:2204.06125 1.2 (2022): 3.

[9] Ajay, Anurag, et al. "Is conditional generative modeling all you need for decision-making?." arXiv preprint arXiv:2211.15657 (2022).

We thank the reviewer for the constructive comments. To further address the concerns of reviewer, we conducted extra experiments and provide the following response.

We used interPlan [1], a closed-loop driving benchmark extended from nuPlan, which contains 335 long-tail traffic scenarios including jaywalkers (Jaywalk), accidents (Nudge Around), and dense-lane-change situations (High Traffic Density) etc, posing a huge challenge in interactive behavior modeling. We achieved SOTA performance on the interPlan benchmark.

*: prior knowledge is used for the model

We conducted further ablation studies on the interPlan by adding the modules onto the base model. We also compared our model with an vanilla imitation learning-based model Vanilla IL, which shares the same model architecture but directly regresses the trajectory, and a Diffusion adaption of our model. The proposed modules bring significant performance improvement, and the result demonstrates the superiority of Flow Matching-based paradigm compared with vanilla imitation learning and Diffusion for interactive behavior modeling.

W1 & W2: Limited Novelty and Unclear Effectiveness of the Interaction-enhanced Spatiotemporal Fusion Block and Scale-adaptive Attention

We propose a coordinated solution for interactive behavior modeling in autonomous driving, with our Interaction-enhanced Spatiotemporal Fusion Mechanism serving as a key component.

• As a subcomponent, our architecture is not naively implemented. Crucially, it coordinates with Fine-grained Trajectory Tokenization to enable richer bidirectional interactions among heterogeneous tokens. Besides, with the help of Scale-adaptive Attention, traffic participants within a certain distance are assigned higher attention scores, enabling each trajectory token to capture spatiotemporal interactions more precisely and adaptively. All these design enhances the expressiveness of interactive behavior modeling while serving as the foundation for flow matching and classifier-free guidance designs.
• Previous methods attempt to enhance interactive behavior modeling by employing complex game-theoretic models [2] or topology-based representations [3] to capture intricate interactive behaviors. In contrast, we pursue an elegant yet effective architecture that demonstrates its effectiveness through our method's SOTA performance (90.43), achieving marked improvement over vanilla Transformer approaches (PlanTF [4], 84.27) and complex specialized architectures (GameFormer [2], 13.32). Our model also achieves SOTA performance (61.82) on interPlan benchmark with the significant performance gain (+14.08) brought about by the two key designs.

W3: Limited Relevance of Flow Matching & Classifier-Free Guidance (CFG) to Interactive Behavior Modeling

• W3.1 Limited Relevance of Flow Matching Generative models demonstrate superior multimodal behavior modeling capabilities, which are fundamental for interactive behavior modeling [5][6]. On top of this, Flow Matching provides more stable training and faster convergence, as noted by the reviewer. As is shown in the table above, when incorporated with our architecture, our method (61.82) outperforms the imitation-learning based model (32.39) and the diffusion adaptation (40.71) with a significant lead in interactive scenarios. Note that the Diffusion adaptation is trained with the same epochs (200 epochs) with Flow Planner, only half the steps of Diffusion Planner [5], indicating the advantage of faster convergence of Flow Matching.
• W3.2 Limited Relevance of CFG While CFG is widely used in diffusion models, our work novelly conditions on neighboring vehicles, effectively enhancing interactive behavior modeling for autonomous driving. Unlike typical CFG use in tasks with explicit conditions (e.g., text prompts [7] or rewards [8]), interaction in driving lacks clear definitions. We find that neighboring vehicles provide a powerful implicit condition. By learning both unconditioned and scene-conditioned behaviors, CFG captures interaction patterns—offering a novel solution for generating interactive behaviors. As shown, our model achieves 61.82 on interPlan vs. 58.22 without CFG, with notable gains in Nudge Around (+8.50) and High Traffic Density (+7.13).

[1] Hallgarten, Marcel, et al. "Can vehicle motion planning generalize to realistic long-tail scenarios?." 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2024.

[2] Huang, Zhiyu, Haochen Liu, and Chen Lv. "Gameformer: Game-theoretic modeling and learning of transformer-based interactive prediction and planning for autonomous driving." Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023.

[3] Liu, Haochen, et al. "Reasoning multi-agent behavioral topology for interactive autonomous driving." Advances in Neural Information Processing Systems 37 (2024): 92605-92637.

[4] Cheng, Jie, et al. "Rethinking imitation-based planners for autonomous driving." 2024 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2024.

[5] Zheng, Yinan, et al. "Diffusion-Based Planning for Autonomous Driving with Flexible Guidance." The Thirteenth International Conference on Learning Representations.

[6] Liao, Bencheng, et al. "Diffusiondrive: Truncated diffusion model for end-to-end autonomous driving." Proceedings of the Computer Vision and Pattern Recognition Conference. 2025.

[7] Ramesh, Aditya, et al. "Hierarchical text-conditional image generation with clip latents." arXiv preprint arXiv:2204.06125 1.2 (2022): 3.

[8] Ajay, Anurag, et al. "Is conditional generative modeling all you need for decision-making?." arXiv preprint arXiv:2211.15657 (2022).