1. Performance test and discussions

We thank the reviewer for highlighting the need for a more thorough analysis and discussion of the experimental results. We note that we have provided a more detailed analysis in the supplementary materials, and we provide additional analysis and discussions below.

First, we would like to clarify that our near real-time inference results were obtained using a single GPU, not multiple GPUs as the reviewer noted. In the following sections, we include a detailed summary of model size, inference speed (in FPS), and hardware configurations, in comparison with other methods.

Further, following the reviewer’s suggestion, we conducted additional experiments to analyze runtime performance under various training settings. The results show that our proposed post-training method (RFT with CoT length penalty) can significantly improve inference efficiency by reducing slow thinking. These results show that RFT effectively shifts model behavior toward faster inference modes without sacrificing planning performance.

We also break down runtimes by thinking mode. As shown in the table below, fast thinking yields much faster and more stable inference compared to slow thinking. Our training approach encourages the model to use fast thinking in simple scenarios and reserve slow reasoning only when necessary.

2. Process of discretizing and tokenizing actions

We thank the reviewer for pointing out the need for greater clarity regarding our action tokenization process. We have provided more implementation details in the supplementary materials. To address the reviewer's comments, we include the following elaborations:

1. Discretization Strategy: We want to emphasize the structural differences between robotic manipulators and autonomous vehicles first. Robot arms are fully actuated, holonomic systems capable of executing independent actions per dimension. However, vehicles represent classical underactuated, nonholonomic systems with three degrees of freedom (i.e., position $x$, $y$, and $heading$) controlled indirectly via acceleration and steering rate. Therefore, we discretize the action space by representing each action token as a short-term, kinematically feasible trajectory segment (0.5 seconds). This tokenized representation encodes motion primitives that are reachable under nonholonomic constraints, serving as building blocks for long-term trajectories.
2. Use of Short-term Spatial Positions: Short-term trajectory segments provide a compact and expressive representation of immediate vehicle movement, which aligns well with the autoregressive generation of our model. This strategy reduces the complexity of long-horizon prediction and improves training efficiency.
3. Why K-Disk Clustering: This method groups similar short-term state transitions into discrete tokens while preserving continuity and ensuring dense coverage of diverse motion patterns across three degrees of freedom (i.e., position $x$, $y$, and $heading$).

[1] Brohan et al. "RT-1: Robotics transformer for real-world control at scale." RSS 2023.

[2] Pertsch et al. "Fast: Efficient action tokenization for vision-language-action models." RSS 2025.

We compared our K-disk tokenization against RT-1 (action bins) and FAST (tokenizing trajectories based on discrete cosine transform) using a fixed codebook size of 2048. The results below report the accuracy of trajectory reconstruction based on ground-truth tokens. Our method achieves the lowest reconstruction error.

To assess downstream impact, we compare our model with the FAST tokenization method (without RFT) on the nuPlan (NAVSIM) test data. The table below shows that our method significantly outperforms FAST in planning metrics. This is largely due to our tokens directly representing physically meaningful vehicle motions, whereas FAST tokens encode more abstract frequency components, which are harder to learn from limited data.

Thus, our discretization and tokenization strategy not only improves trajectory reconstruction accuracy but also translates into better planning performance.

3. Analysis of benchmark results

We thank the reviewer for their feedback on the significance of our results. While we acknowledge that our model does not achieve absolute SOTA performance, we would like to clarify several key aspects of our framework that we believe underscore its contribution and broader significance.

• Generality and Simplicity. Unlike prior works that are often optimized for specific benchmarks and benefit from privileged modalities such as LiDAR, HD maps, and object-level annotations, our framework is designed to be general, scalable, and end-to-end, relying only on camera inputs, language, and trajectory supervision. We have listed the comparison of different methods in the nuPlan (NAVSIM) benchmark in the table below. Despite the absence of such privileged inputs, our model achieves competitive results in the benchmark without dataset-specific engineering. In addition, our model is competitive across multiple datasets and tasks (nuPlan, nuScenes, Waymo, and CARLA). We consider this cross-domain consistency a key contribution. Our results are also promising and suggest the viability of a unified formulation of planning and reasoning in autonomous systems.
• Scalability with Data. We observe clear data scaling trends in our experiments, indicating that model performance improves substantially with increased training data. Due to the limited size of available open-source datasets, we believe that the full potential of our approach has not yet been realized.

Best-of-N Analysis and RFT. The best-of-N planning evaluation is intended to illustrate the performance ceiling achievable by leveraging the model’s inherent trajectory diversity. Although competing baselines do not include best-of-N evaluation, we emphasize that our RFT method achieves performance close to this upper bound while operating in a single-shot manner.

4. Novelty

We appreciate the reviewer for the opportunity to clarify the contributions and insights presented in our work. We respectfully argue that the originality of our paper lies not only in these action tokenization methods but also in the following key contributions:

1. Unified Vision-Language-Action Planning Framework: Our framework bridges vision, language, and action in a unified learning setup for autonomous driving tasks, where discrete, physically-plausible trajectory tokens form the basis for autoregressive prediction. Although individual components (e.g., trajectory learning or language reasoning) have been studied in isolation, their integration under the VLA paradigm is novel and offers new potential for end-to-end, generalizable planning.
2. Reinforcement Fine-Tuning for Trajectory Planning: To the best of our knowledge, this is the first application of GRPO for trajectory-level policy refinement in autonomous driving with vision-language models. While RFT has been extensively explored in natural language generation, our work demonstrates that it can also be effectively applied to trajectory prediction and refinement, yielding measurable improvements in planning quality. This establishes a new direction for aligning planning policies with preference-based or task-driven feedback.
3. Fast-Slow Planning Paradigm: We provide new insights into how SFT (mixture thinking-mode training) and RFT can be leveraged to enable an adaptive fast-slow planning paradigm for VLA models. Empirically, this strategy improves overall planning performance and significantly enhances runtime efficiency, as the fast-thinking mode can suppress language-based reasoning outputs in straightforward scenarios, but keep reasoning capabilities in challenging situations.

Thank you once again for the additional clarification. While we understand that you are finalizing your review, we appreciate the opportunity to respond now that your specific concerns are more clearly stated. We would like to respectfully highlight several points of clarification that we believe underscore the novelty and significance of our work. Our intention was not to reiterate without substance, but to ensure key contributions were properly communicated. In this response, we aim to provide clearer differentiation from prior work and emphasize the technical insights of our approach.

1. Action Tokenization in Robotics vs. Autonomous Driving

Discrete tokenization has become a common technique in VLA models to integrate actions into VLMs for outputting control commands. This approach has been used in robotics, as seen in works like RT-1 [1], OpenVLA [2], and FAST [3]. Although autonomous vehicles are a specialized form of robotics, they differ significantly in terms of system dynamics and control constraints. Specifically, autonomous driving involves nonholonomic, underactuated systems with different requirements than robotic manipulators.

We show that tokenization methods developed for robotics do not perform well in autonomous vehicles. Therefore, we utilize a domain-specific tokenization scheme (K-Disk) that accounts for the vehicle’s kinematics and trajectory feasibility. As shown in our experiments, this results in lower reconstruction errors and better downstream planning performance compared to RT-1 and FAST tokenization. We believe this domain adaptation and its impact represent a meaningful novelty in VLA models for autonomous vehicles.

[1] RT-1: Robotics Transformer for Real-World Control at Scale. RSS 2023.

[2] OpenVLA: An Open-Source Vision-Language-Action Model. CoRL 2024.

[3] FAST: Efficient Action Tokenization for Vision-Language-Action Models. RSS 2025.

2. Fast-Slow Planning Paradigm: Implementation Difference

The fast-slow thinking paradigm is an established concept in cognitive science and AI systems [4]. We referred to one previous work (DriveVLM [5]) using this concept for autonomous vehicles. However, we emphasize that our implementation is significantly different from it. In DriveVLM, fast and slow processes are handled by two separate models, for example, a conventional BEV-based system for the fast component and a VLM for the slow component. This dual-model approach can introduce conflicts when switching between modes, require more training resources, and have significant engineering complexity.

In contrast, our method integrates both fast and slow thinking within a unified autoregressive framework. We also use RL post-training to enable the model to improve its behaviors and adaptively switch between modes based on reward feedback. This unified design not only simplifies deployment but also provides improved performance and inference efficiency.

[4] Thinking fast and slow in AI. AAAI 2021.

[5] DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Models. CoRL 2024.

3. Effective Training Paradigm and Use of RFT

While post-training of VLMs/LLMs using GRPO-style reinforcement learning has been explored in language generation, to the best of our knowledge, we are the first to apply reinforcement fine-tuning to trajectory-level planning in VLA models for autonomous driving.

Our results show that RFT significantly improves planning performance and enables behavior shaping (e.g., encouraging fast thinking where possible). This adaptation of a language modeling paradigm to the AD domain is non-trivial and demonstrates practical utility.

4. Generality and Cross-Domain Generalization

We clarify that the generality of our approach is demonstrated by the ability of a single model to work effectively across multiple datasets and benchmarks (nuPlan, Waymo [Vision-only End-to-End Driving], nuScenes, CARLA [Closed-Loop Driving]) without requiring architecture changes or domain-specific tuning. The same model generalizes to different environments, tasks, and datasets, which is rarely achieved or showcased in existing end-to-end driving models. While we rely on camera input, this is a deliberate design choice to ensure scalability and accessibility. Prior methods often rely on LiDAR, HD maps, or object-level annotations, which we explicitly avoid to ensure generality across varied datasets and settings.

We sincerely hope that these clarifications help better communicate the contributions and novelty in our work. Again, we appreciate the time and consideration the reviewer has given to our work. We hope that the above clarifications help position our contributions in a more precise light, even if they do not change the final evaluation.

Thank you again for your time and consideration.

1. Evaluation on language-related benchmarks

We thank the reviewer for the suggestion of evaluating the model’s reasoning ability on language-related benchmarks. While we agree that this could offer additional insights, we would like to clarify that our model is not trained for VQA tasks typically assessed by DriveLM. Instead, we repurpose the DriveLM dataset to enable structured CoT reasoning in support of action planning. As a result, direct evaluation using the DriveLM benchmark is not well aligned with the functions of our model.

Moreover, language reasoning performance in our setting is primarily determined by the underlying VLM and its training corpus. Thus, it does not serve as a key differentiator in our work. Our contribution lies in augmenting VLMs with planning capabilities for autonomous driving, rather than improving general language understanding.

2. Evaluation of RFT on reasoning and planning

We thank the reviewer for raising this question regarding the trade-off between reasoning depth and efficiency in the RFT process.

To address the reviewer’s concern, we clarify that the CoT penalty during RFT is applied using a sigmoid-based normalization function, with a tolerance length $L_{tol}$ calibrated based on the average length of reasoning annotations in our training data. This design ensures that excessively long CoT outputs receive meaningful penalties, while preserving the flexibility for necessary CoT reasoning. The goal is to improve inference latency in time-sensitive planning tasks.

To evaluate the trade-off, we conducted an ablation study comparing the model’s performance with and without the CoT penalty. Due to the time limitation during the rebuttal period, we finished 1,500 training steps, and we compared the model with the same setting with a CoT length penalty. The results in the table below show that adding a CoT length penalty in RFT results in better planning performance and significantly reduces inference latency.

We will include expanded results and analysis in the revised paper.

3. Comparison with stronger methods on CARLA

We thank the reviewer for pointing out the need for more comparisons. We compare AutoVLA with methods trained on PDMLite-gathered datasets. We also carefully tuned the low-level controller, which we found to be a critical factor in tracking the planned trajectory. As a result, AutoVLA’s driving score improved from 78.84 to 80.12. Our updated results demonstrate that AutoVLA is comparable to these state-of-the-art models.

Specifically, while Transfuser++ employs LiDAR inputs and multimodal feature fusion, AutoVLA is a vision-only model. Similarly, SimLingo utilizes approximately 650k samples from different sources, whereas AutoVLA is trained with only 289k samples. Despite these differences, AutoVLA achieves comparable performance in closed-loop driving.

4. Computational cost and inference latency

We acknowledge the concern regarding inference latency, which may be a limiting factor of our current system. However, we note that the current system has not been optimized for runtime efficiency, as our primary focus in this work is to demonstrate the model's reasoning and planning capabilities. We plan to apply model quantization and other techniques to improve inference speed, which have already been widely explored in LLM. We believe that with such runtime optimizations, the model can achieve near real-time performance even with slow reasoning.

We report more latency statistics over 1,500 test samples from the nuPlan dataset. The results show that the base SFT model relies more heavily on slow thinking, resulting in higher latency. Our proposed RFT method effectively reduces unnecessary slow reasoning while preserving its use in complex cases, leading to significantly improved runtime efficiency.

Additionally, we provide a breakdown of runtimes by thinking mode. The results show that fast thinking is significantly faster and more stable than slow thinking.

5. Teacher model

We appreciate the reviewer’s suggestion to provide more details regarding our teacher model. Detailed descriptions of the teacher model and visualizations of the reasoning data annotations are included in the Supplementary Material. Our teacher model incorporates ego vehicle status (velocity, acceleration, historical actions) as language-based inputs.

Due to Rebuttal time constraints, we randomly selected 60 Waymo scenarios to conduct an ablation study comparing inference accuracy with and without ego status input. Human verification revealed that excluding ego status decreased accuracy from 86.7% to 78.3%. Supplementary Material Figure S3 (scenario 1) exemplifies a situation where ego states are essential; specifically, the vehicle has been stopped at a stop sign, and it can accelerate and execute a left turn, rather than stop at the stop sign.

Furthermore, our method integrates ground-truth driving actions as explicit hints to guide the generation of causal explanations. These ground-truth actions are generated by manually defined rules based on ground-truth trajectories. Consequently, the final decisions of our model can strictly adhere to ground-truth driving actions, achieving a human-verified accuracy of 93.6% across a randomly selected set of 3,862 Waymo scenarios. The error comes from the inherent limitation of the rule-based approach, particularly in handling complex multi-stage behaviors commonly encountered in challenging Waymo scenarios, such as accelerating from a stop sign and subsequently decelerating for pedestrians. Incorrect predictions identified during human verification are manually removed or revised to ensure accuracy.

1. Robotic scenario

We thank the reviewer for raising this point. While our work focuses on autonomous driving, the underlying ideas of AutoVLA, are indeed inspired by and relevant to broader robotic scenarios.

We would like to clarify that the motivation for our action tokenization arises from the unique challenges posed by vehicle dynamics, which differ significantly from typical manipulation robots (e.g., RT-1 [1], FAST [2]). Most robotic platforms in those settings are holonomic and fully actuated, allowing for independent per-dimension action control at each timestep. In contrast, vehicles are underactuated, nonholonomic systems with kinematic constraints that prevent arbitrary motion, such as instantaneous lateral movements.

To address this, our action tokenization method encodes short, dynamically feasible trajectory segments in terms of position and heading, enabling smooth and physically consistent vehicle motion. Nevertheless, we agree that evaluating AutoVLA in general robotic scenarios would further highlight the broader utility of the proposed framework. Extending our framework to robot navigation represents a promising direction, and we plan to pursue this in future work.

[1] Brohan et al. "RT-1: Robotics transformer for real-world control at scale." RSS 2023.

[2] Pertsch et al. "Fast: Efficient action tokenization for vision-language-action models." RSS 2025.

2. Action codebook size

We appreciate the reviewer’s question regarding codebook size. The codebook size $K=2048$ used in our AutoVLA framework was selected based on empirical evaluations that balance trajectory reconstruction accuracy with training stability. We conducted comparative evaluations of reconstruction accuracy using various codebook sizes across multiple tokenization methods, including RT-1, FAST, and our K-disk tokenization. The results demonstrate that our K-disk method consistently achieves the best reconstruction accuracy across different codebook sizes.

When the codebook size is small ($K=256$), the reconstruction error is significantly higher due to limited expressiveness. As the codebook size increases beyond 1024, the improvements in accuracy become marginal, and the risk of redundant or overlapping motion patterns also increases. To strike a balance between accuracy and codebook efficiency, we selected $K=2048$ as a practical choice.

We also evaluated downstream performance using the PDMS metric on the nuPlan (NAVSIM) test data. Our results show that $K=2048$ yields the highest score among the testing configurations. While increasing the size from 1024 to 2048 leads to an improvement, expanding the size from 2048 to 4096 yields a marginal decrease. The result indicates that a codebook size of 2048 is sufficient to capture the majority of meaningful motion primitives, and that a larger codebook size introduces increased redundancy, making further expansion unnecessary.

Based on analyses of trajectory reconstruction accuracy and driving performance across various $K$ values, we selected $K=2048$ as our codebook size.

Lastly, we emphasize that the discrete trajectory segments encoded in our codebook are derived from real-world driving data, inherently satisfying vehicle kinematic constraints. This ensures that the learned tokens are not only expressive but also physically feasible.

3. Quantitative results for the fast-slow system

We thank the reviewer for highlighting the need for a more in-depth analysis of the fast-slow thinking system. We have conducted additional experiments and provided detailed quantitative results to support the effectiveness of our approach. The advantage of the fast-slow thinking system is its ability to maintain or improve planning performance while significantly reducing unnecessary slow reasoning, thereby enabling efficient inference. Our results in the main paper demonstrate that the RFT with the CoT length penalty training achieves better planning performance while substantially reducing average inference time.

We add results for RFT without the CoT length penalty, which shows slightly worse planning performance and inferior runtime efficiency due to excessively slow reasoning. This highlights the advantage of our training method for fast-slow systems: by incorporating a CoT length penalty, we guide the model to use slow reasoning only when necessary, achieving better overall running efficiency.

We further analyze the distribution of runtimes and mode usage across test samples. The results demonstrate that our proposed post-training method (RFT w/o CoT length penalty) significantly improves inference efficiency by reducing reliance on slow thinking while preserving reasoning capability when needed.

We also provide a breakdown of runtime by thinking mode across the testing samples. Fast thinking is significantly faster and more stable than slow thinking, while slow thinking, though more time-consuming, can be kept for complex scenarios.

These results demonstrate the advantage of our fast-slow thinking system. The ability to switch between fast and slow thinking enables AutoVLA to respond efficiently in straightforward scenarios while retaining the capacity for deeper reasoning when required.

We thank the reviewer for the positive evaluation and are glad that our responses have addressed your concerns. We agree that robotic experiments would further strengthen our work. Due to current hardware and integration constraints, we are unable to include them at this stage. We consider this an important next step and plan to conduct real-world robotic deployments as part of our future work. We greatly appreciate your time and thoughtful feedback throughout the review process.

1. Action tokenization

We thank the reviewer for the comment regarding the evaluation of our action tokenization method. As suggested, we compare our tokenization approach with other advanced tokenization methods used in robotic control, including action bins in RT-1 [1] and the DCT-based method in FAST [2]. The table below summarizes the accuracy of each method. Specifically, we evaluate trajectory reconstruction accuracy (ADE/FDE) across 120K trajectories from the nuPlan dataset, using a consistent codebook size of 2048.

[1] Brohan et al. "RT-1: Robotics transformer for real-world control at scale." RSS 2023.

[2] Pertsch et al. "Fast: Efficient action tokenization for vision-language-action models." RSS 2025.

It is important to emphasize the structural differences between robotic manipulators (e.g., in RT-1 and FAST) and autonomous vehicles first. Robotic arms are typically fully actuated, holonomic systems capable of executing independent actions per dimension at each timestep. However, vehicles represent classical underactuated, nonholonomic systems with three degrees of freedom (i.e., position $x$, $y$, and $heading$) controlled indirectly via acceleration and steering rate. These systems are constrained by kinematics and cannot perform arbitrary motions. Therefore, our approach instead represents actions as short, feasible trajectory segments, encompassing position $x, y$, and $heading$, enabling the generation of smoother and physically feasible trajectories for vehicle systems.

For the RT-1 tokenization method, we discretize acceleration and steering rate using uniform action bins and apply a kinematic model to reconstruct the long-term trajectory. However, since we only have trajectory-level data and we have to indirectly derive control actions from it, this binning approach leads to the highest reconstruction error. In contrast, our method achieves the lowest ADE and FDE, demonstrating its ability to represent the trajectory accurately. FAST method performs comparably in terms of reconstruction accuracy, but it uses variable-length token sequences (typically 1 to 25 tokens per trajectory), which complicates the selection of appropriate token lengths and makes fixed-horizon trajectory prediction more challenging.

We also evaluate planning performance using the FAST tokenization method (without RFT) on the nuPlan (NAVSIM) data. The table below shows that our method outperforms FAST in planning metrics, including PDMS, No At-fault Collision, and Ego Progress. This is largely due to our tokens directly representing meaningful vehicle motions, whereas FAST tokens encode more abstract frequency components, which are harder to learn from limited data.

2. RFT in closed-loop experiments

We thank the reviewer for the insightful comment and agree that closed-loop RFT would provide a better assessment of long-term planning performance. However, closed-loop fine-tuning using existing open-source driving simulators (e.g., CARLA) is computationally prohibitive due to their limited simulation throughput and system bottlenecks. Furthermore, the primary goal of our RFT approach is to improve trajectory planning accuracy and running efficiency, rather than to explicitly address robustness to distributional shifts. In future work, we plan to enhance the model architecture and develop more efficient simulation environments to enable closed-loop RFT and validate its capability.

3. Benchmark evaluation and significance

We appreciate the reviewer’s concern regarding the significance of our proposed framework. We need to clarify that the primary goal of our work is to propose a general and scalable framework that can operate across a wide range of autonomous driving tasks, rather than optimizing for a single benchmark or dataset. Accordingly, our evaluations are intentionally broad, covering diverse datasets such as nuScenes, nuPlan, Waymo, and CARLA, without relying on dataset-specific engineering or privileged modalities. Our method consistently demonstrates competitive results across different datasets and benchmarks, which we argue is a meaningful indicator of its generalization ability.

It is important to note that many competing methods in the benchmarks are specialized and benefit from LiDAR sensors or privileged information (e.g., object-level annotations and map priors). In contrast, our approach is purely end-to-end, relying only on raw visual inputs and language and action supervision, which aligns with our goal of developing a more unified and scalable autonomous driving paradigm.

Moreover, our results reveal clear data scaling trends: the performance of our model improves with increased data, suggesting that its full potential has not yet been realized under the current limitations of available open-source datasets.

Comparison of model sizes and FPS

Many baseline methods do not report model parameters or FPS, making direct comparison difficult. We have included our model's size and runtime performance in the table below, and will add those results in the revised paper. We acknowledge that model size and inference speed are limitations. However, we note that the current system has not been optimized for efficiency, as our primary focus is to demonstrate the model's reasoning and planning capabilities. We also show that RFT enables adaptive thinking modes and improves performance and inference speed.

Notably, our training method is also applicable to smaller VLMs, which can improve runtime performance in deployment. We also plan to explore LLM quantization in future work to enable near real-time inference even with slow reasoning.

4. CoT training in nuScenes

Thank you for pointing out the performance degradation with CoT training. After careful examination, we found that this issue comes from the generation parameters used across different settings. For models trained with CoT, we adopted more diverse and random generation parameters (Temp=0.8, Top-p=0.5, Top-k=20) to encourage more reasoning outputs. In contrast, the action-only models employed more deterministic settings (Temp=0.01, Top-p=0.001, Top-k=1). However, this diversity-promoting generation parameter introduces increased randomness in inference, which negatively impacts trajectory accuracy metrics, such as L2 error on the nuScenes dataset.

We conducted an additional experiment to examine the influence of generation parameters. The results indicate that when CoT models are evaluated using the same generation parameters as the action-only models, the performance becomes comparable, with no significant difference. We will revise the paper accordingly to clarify this explanation and include the new experimental results.

Regarding the RFT in nuScenes, the core of RFT is verified reward functions. Unlike traditional displacement-based metrics (e.g., ADE), which merely enforce adherence to reference trajectories, the reward function necessitates comprehensive evaluation encompassing comfort, safety, and efficiency. The NAVSIM platform within nuPlan provides such a comprehensive evaluation platform with the PDMS metric. Conversely, nuScenes (designed for perception tasks) lacks a comparable planning evaluation framework, limiting the exploration of RFT. Additionally, nuPlan provides a larger and more diverse driving dataset (178K data), whereas nuScenes contains only 25K data from two similar urban locations as nuPlan (Boston and Las Vegas). Therefore, we argue that RFT's performance benefits can generalize from nuPlan to nuScenes dataset.

We sincerely appreciate your feedback and your recognition of our efforts in addressing the earlier concerns. We respectfully respond to the remaining points regarding the significance and potential impact of our work, with the aim of further clarifying the contributions of our framework.

1. Generalization and Dataset-Specific Training

While we fine-tune the model on individual datasets to improve performance for specific benchmarks, we emphasize that the architecture, training method, and action tokenization remain consistent across datasets. This stands in contrast to many existing approaches that require extensive dataset-specific engineering, such as custom sensor fusion modules, object detection pipelines, or HD map integration.

In our work, a single unified model is evaluated across diverse datasets and benchmarks (including nuPlan, Waymo Vision-only End-to-End Driving, nuScenes, and CARLA Closed-loop Driving) without architectural modifications or domain-specific tuning. This level of cross-domain generalization is rare among end-to-end driving models, and significantly reduces engineering effort while enhancing scalability.

2. Cost-Effectiveness and CoT Training

Although CoT training still introduces a form of supervision, it is fundamentally different from object-level annotations or HD maps, which require costly manual labeling and dedicated infrastructure.

• Our CoT supervision is generated automatically using capable vision-language models, with only minimal human verification of textual descriptions, rather than manual annotation across different tasks and sensor configurations.

• Notably, language (CoT) can be a general abstraction of driving perception and policy, allowing adaptation across various driving scenarios and sensor modalities.

• Furthermore, our framework effectively integrates language-based instructions, aligning world knowledge in large language models with physical driving actions.

We show that this leads to improved performance in long-tail scenarios, and achieves the highest RFS (Spotlight) score on the Waymo End-to-End Driving benchmark, which contains the most challenging cases.

3. Significance and Potential Impact

We acknowledge the current limitation in inference speed and have discussed it explicitly. However, we would like to emphasize that this work is an important step toward bridging large vision-language models with trajectory planning for autonomous driving, unifying perception, reasoning, and planning in a single model.

• Our RFT training paradigm shows strong potential for improving both planning performance (11% increase in planning score) and efficiency (67% decrease in runtime), and for grounding physical actions in language-based reasoning.

• Additionally, our results show clear data scaling trends, suggesting further performance gains with larger and more diverse datasets.

We believe these aspects indicate substantial long-term promise despite current limitations.

We hope these clarifications help better convey the significance and contributions of our work. We would be very grateful if you would consider these additional points in evaluating your assessment.

Thank you again for your time and consideration.

We sincerely thank the reviewer for acknowledging the contributions of our work and for the positive feedback. We are committed to addressing the current limitations, particularly in runtime and deployment efficiency. We greatly appreciate your time, constructive comments, and engagement throughout the review and discussion process.