RLGF: Reinforcement Learning with Geometric Feedback for Autonomous Driving Video Generation

We are very grateful for your expert review, which has pushed us to conduct crucial new experiments that we believe significantly strengthen our paper. We are happy to share the new results below.

Regarding Q1: Synthetic+real dataset setting. Following your suggestion, we conducted this crucial experiment, aligning with DiVE and Glad. Table 5 below shows that augmenting real data with our RLGF-enhanced synthetic data provides a significantly larger boost to downstream performance than using baseline synthetic data.

Table 5: Impact of augmenting real data with synthetic data.

Regarding Q2: On the Completeness of StreamPETR Results. We apologize that the original table was incomplete and have run the full evaluation on StreamPETR (results below), including other SOTA baselines for a comprehensive comparison. Our key finding remains: RLGF provides a +4.75% mAP gain over the strong DiVE baseline. We agree that models like GLAD show strong performance. We wish to clarify that RLGF is a plug-and-play refinement technique designed to improve any diffusion-based generator. We see our work as complementary, not competitive with SOTA generators. We will apply RLGF to Glad as soon as it releases.

Table 6: Detailed 3D Object Detection (3DOD) performance on nuScenes validation using BevFusion and StreamPETR.

Regarding Q3: Dependence on perception models. This is a fair point. We tested robustness by using a "weaker" perception model (with 30% training data and a 25% performance drop). RLGF still improved downstream mAP by +4.2%. This demonstrates that our framework is robust, though we agree a better perception model yields better results. We will add this analysis to the appendix.

We thank you for your positive evaluation and for questions that help clarify our work.

Regarding Q1: Why not use a detection-based reward? We tested this experimentally and found it to be ineffective. Our choice of reward model is a cornerstone of our work—a deliberate design decision based on principles of signal density, stability, and directness, which we validate with the following three key points.

• a) Why Geometric Rewards: Our core hypothesis is that flawed 3D geometry is the root cause of poor downstream performance. Instead of using a complex, indirect signal, we directly target this issue with our HGR. This multi-level system evaluates point-line-plane alignment and scene-level coherence, providing a dense and stable reward signal. This structure enables fine-grained feedback on specific geometric aspects, which is crucial for effective and stable RL training.
• b) Why Not a Detector Reward: We experimentally tested this intuitive alternative. As shown in the Table 1 below, using a 3D detector as a reward is far less effective. This is because detector-based rewards are sparse (only rewarding on detected objects) and suffer from unstable gradients (due to non-differentiable operations like NMS), providing a poor learning signal for the diffusion model.
• c) Robustness to Downstream Detectors: To demonstrate generality, we evaluated our method on the stronger StreamPETR detector (appendix). RLGF achieved a substantial +4.75% mAP gain, proving our geometric improvements are robust and benefit diverse downstream architectures.

Table 1: Effectiveness of reward signals.

Regarding Q2: Relationship with RL. Thank you for the suggestion. We will add a more intuitive explanation: the diffusion model is an Agent that learns to take noise-prediction Actions to maximize a geometric correctness Reward from our perception-model Environment.

Thank you for your thoughtful feedback on the practical aspects of our work.

Regarding Q1: Computational Complexity. Our Latent-Space Windowing Optimization is the key to making our approach computationally feasible. As the new Table 3 shown below, full rollouts in latent or image space lead to Out-of-Memory (OOM) errors. Our method requires 28 hours of fine-tuning on 8 A100 GPUs, which represents only about 15% additional training time over the baseline. We will add this analysis to the appendix.

Table 3: Computational cost analysis.

Regarding Q2: Robustness to inaccurate perception models. This is a critical point. To directly test robustness, we intentionally trained a "weaker" perception model (using only 30% of training data, resulting in a ~25% performance drop). Even with this degraded reward signal, RLGF still boosted the downstream mAP by a significant +4.2%. This demonstrates our framework's robustness. We will add this new study to the appendix.

Regarding Q3: Generalization beyond nuScenes. To demonstrate generalization, we built a new pipeline on the CARLA with Dive. On a baseline of 57.6% mAP, applying RLGF provided a +5.4% mAP improvement. This confirms that our method's principles are not dataset-specific.

Table 4: Generalization results on the CARLA

Regarding Q4 & Q6: Ablations and clarifications. We have noted in the appendix that our parameters were chosen based on empirical experiments, and we provide the detailed ablation results here for clarity, including the RL window size, reward weights, and the starting step of the sliding window. All ablations are performed on the DiVE baseline and evaluated with BEVFusion.

Thank you for your positive and constructive review. We are delighted you found our contributions valuable and address your questions below.

Regarding Q1 & Q4: On the choice and robustness of reward models. Thank you for these excellent and related questions. Our choice of reward model is a cornerstone of our work. It was a deliberate design decision based on providing a reward signal that is dense, stable, and directly addresses the core problem. We validate this approach with the following three key points.

• a) Why Geometric Rewards: Our core hypothesis is that flawed 3D geometry is the root cause of poor downstream performance. Instead of using a complex, indirect signal, we directly target this issue with our HGR. This multi-level system proposes point-line-plane alignment and scene-level coherence, providing a dense and comprehensive reward signal. This structure enables fine-grained feedback on specific geometric aspects, which is crucial for effective and stable RL training.
• b) Why Not a Detector Reward: We experimentally tested this intuitive alternative. As shown in the new Table 1 below, using a 3D detector as a reward is far less effective. This is because detector-based rewards are sparse (only rewarding on detected objects) and suffer from unstable gradients (due to non-differentiable operations like NMS), providing a poor learning signal for the diffusion model.
• c) Robustness to Downstream Detectors: To demonstrate generality, we evaluated our method on the stronger StreamPETR detector (appendix). RLGF achieved a substantial +4.75% mAP gain, proving our geometric improvements are robust and benefit diverse downstream architectures.

Table 1: Effectiveness of reward signals.

Regarding Q2: Temporal Consistency. Temporal consistency is primarily enforced by our Latent Occupancy Prediction Model ($P_\text{occ}$), which explicitly operates on video sequences. We will clarify this in Section 3.4.

Regarding Q3 & Q5: Ablation Study on Hyperparameters. We provide detailed ablation results in the new Table 2 below, which validate our choices for key hyperparameters. The results show that a window size of w=5 and our balanced reward weights are optimal. We will add this to the appendix.

Table 2: Ablation Study about Hyperparameters

Regarding W3: Limitation & Future Work. We have expanded the "Limitations and Future Work" section in the appendix (Section C).