Thank you for the thoughtful review and valuable feedback. We are glad that “the applicability and methodology of our work” are recognized as “good”. We now address the reviewer’s concerns regarding the framing of our method, particularly on our use of LiDAR.

To clarify this, we (1) explain the dependency on LiDAR for baselines and provide additional experiments demonstrating that Flux4D can also operate in a LiDAR-free setting, by using off-the-shelf monocular depth estimation as an alternative, and (2) argue that using LiDAR should not be considered as a form of supervision in the context of self-driving applications, where LiDAR is commonly available and passively collected.

Q1: Dependence on LiDAR modality – Comparisons with previous works makes it hard to evaluate the scientific contributions
A1: We acknowledge that our method leverages LiDAR to initialize our Gaussian scene representation and uses it for depth supervision. However, we would like to point out that all of our baselines (in Tab. 1, 2, 4) also have access to this data either in reconstruction or during training. Specifically, EmerNeRF, Desire-GS, G3R, StreetGS and NeuRAD all use LiDAR in their proposed reconstruction frameworks, two of which also predict scene flows (EmerNeRF, Desire-GS). For general feed-forward generalizable methods (e.g., DepthSplat, L4GM), we believe we have made considerable effort towards incorporating LiDAR in their frameworks (L219-222 and Supp. 105-123). While STORM and DrivingRecon use LiDAR during training as depth supervision, we acknowledge they do not use it at test time.

To demonstrate Flux4D in a LiDAR-free setting at test time, similar to DrivingRecon and STORM, we replaced LiDAR point cloud initialization with metric depth estimated from foundational model Metric3D [1]. Specifically, we removed LiDAR points entirely and instead voxelized the projected monocular depth with a 5 cm resolution, retaining up to 2M points, and then trained the model following the same 3D processing pipeline. Additionally, as suggested by Reviewer QsPn, we also include some representative scene flow metrics, namely Threeway-EPE (EPE-3way) [2] and Bucketed Normalized EPE (Mean S, Mean D) [3] to better evaluate the motion models.

We observed that the flow estimation performance remains comparable with the original Flux4D, and in some cases, the visual realism improves in background regions (e.g., buildings) due to the broader coverage provided by monocular depth. Moreover, this LiDAR-free version of Flux4D performs significantly better than STORM. This demonstrates that our unsupervised learning of flow and appearance directly in 3D space with an approximate geometry initialization to be the key scientific contribution (as recognized by Reviewer QsPn), rather than the use of LiDAR. We report the numbers below and will include the quantitative results, visualizations, and further analysis in the revision.

Q2: Unsupervised nature of Flux4D and framing
A2: Thank you for raising this point. LiDAR is widely available and passively collected in real-world autonomous driving systems. Moreover, LiDAR in our framework is used purely as raw, sparse sensor input, without any human labels, semantic annotations, or ground-truth flow. The learning is driven entirely by photometric and geometric reconstruction losses, and Flux4D does not rely on supervised labels at any stage. This is consistent with definition and common practice in unsupervised learning, where additional modalities (e.g., lidar, stereo or depth) can be used as inputs and help define an energy or reconstruction objective [4, 5, 6]. Importantly, while LiDAR is sparse, Flux4D produces dense, photorealistic 3D reconstructions. We also note that existing unsupervised per-scene reconstruction methods for driving scenes (DesireGS, EmerNerf, UnIRe [7]) also require the usage of LiDAR during their reconstruction process.

We also provide additional experiments in Q1/A1 demonstrating that Flux4D also performs well without LiDAR, using monocular depth estimated by an off-the-shelf model. We emphasize that this is fundamentally different from methods relying on foundation model priors as supervision, as such supervision can introduce inaccuracies and is not suitable for scalable unsupervised training. The key contribution of Flux4D lies in learning motion and scene dynamics directly from raw RGB and LiDAR inputs, without any annotated supervision. We will clarify this framing in the revised manuscript and more explicitly discuss/emphasize the role of LiDAR within our unsupervised learning setup.

We look forward to further discussions if you have any further questions or concerns.

[1] Hu et al., Metric3d v2: A versatile monocular geometric foundation model for zero-shot metric depth and surface normal estimation, 2024.
[2] Chodosh et al., Re-Evaluating LiDAR Scene Flow for Autonomous Driving, 2024.
[3] Khatri et al., I Can't Believe It's Not Scene Flow!, 2024.
[4] Liu et al., Unsupervised Learning of Scene Flow Estimation Fusing with Local Rigidity, 2019.
[5] Li et al., Neural Scene Flow Prior, 2021.
[6] Vedder et al., Neural Eulerian Scene Flow Fields, 2025.
[7] Mao et al., UnIRe: Unsupervised Instance Decomposition for Dynamic Urban Scene Reconstruction, 2025.

We thank the reviewer for taking precious time in checking our responses. We hope our answers and additional results address your concerns well. Specifically,

Q1/A1: We clarified baseline LiDAR dependencies and provided LiDAR-free Flux4D results.
Q2/A2: We clarified the unsupervised nature of Flux4D.

Please let us know if you have any additional questions. We will be more than happy to clarify them. Any follow-up discussions are highly appreciated!

Thank you for the follow-up discussion! We are glad to hear that the additional results using off-the-shelf monocular depth help strengthen the framing of the paper. We appreciate the framing suggestion to emphasize that our approach leverages general geometric priors directly in 3D space to reduce ambiguity in reconstruction and flow estimation. We will revise our manuscript to better highlight this point.

We also agree that the quality of the geometric prior influences the final performance: LiDAR + monocular depth slightly outperforms LiDAR-only, which in turn is slightly better than monocular depth alone. That said, our method remains highly robust even under imperfect Metric3D depth, thanks to training across diverse scenes and the use of iterative refinement. We will add discussions to better highlight this distinction and improve clarity.

Please let us know if there are any additional concerns, questions, or suggestions, we would be happy to address them.

Thank you for the thoughtful review and valuable feedback. We are pleased that Flux4D was recognized as “simple and effective,” delivering “faster inference” with “better results,” and that its ability to predict flow without supervision was found “impressive”. As suggested, we have added quantitative comparisons with STORM, highlighted our ablation to isolate the impact of the iterative refinement, and will include additional results and analysis in the revision.

Q1: Quantitative comparison with DrivingRecon and STORM
A1: Thank you for pointing this out. We have conducted additional experiments to compare with STORM. The comparisons with DrivingRecon and STORM are summarized as follows.

Comparison to DrivingRecon: We report quantitative comparison with DrivingRecon on WOD following their setting (Line 213–215) in Table 3 (main paper) and provide visual comparisons (Supp Fig. A4). We observe significant improvements by Flux4D (+5.99 dB in PSNR and +0.21 in SSIM), demonstrating the effectiveness for unsupervised dynamic scene reconstruction.

Comparison to STORM: At the time of submission (May 15), STORM was not open-sourced. Following its release afterwards (May 19), we retrained STORM under the same settings as used in Table 1 of the main paper. Additionally, as suggested by Reviewer QsPn, we also include some representative scene flow metrics, namely Threeway-EPE (EPE-3way) [1] and Bucketed Normalized EPE (Mean S, Mean D) [2]. We observe significant improvements of our method in reconstruction realism and flow estimation. Specifically, Flux4D is able to capture high-speed moving actors (e.g., oncoming actors) and small objects / far-away objects thanks to our architectural design to learn flow and appearance directly in 3D space with an approximate geometry initialization (reduce ambiguity and handle full-resolution $1080 \times 1920$ inputs). Please see Q2/A2 for more discussions. We will provide more visualizations and analysis in the revision.

Legend: _D = Dynamics only

Q2: Directly processing Gaussians in 3D space vs predicting pixel-aligned Gaussians in DrivingRecon and STORM
A2: While common in recent feed-forward reconstruction methods [3,4], predicting pixel-aligned Gaussians introduces substantial memory and computational overheads, especially in large-scale driving scenes. This design forces methods such as STORM and DrivingRecon to significantly downsample the image (e.g., ×8 in STORM and ×5 in DrivingRecon), limiting reconstruction fidelity and scalability. In contrast, Flux4D is designed to support high-fidelity, full-resolution ($1920 \times 1080$) reconstruction and simulation. To this end, it adopts a fundamentally different architecture that learns flow and appearance directly in 3D space with an approximate geometry initialization. This 3D design yields a more compact and geometrically consistent representation across views, improving scalability and enabling explicit multi-view flow reasoning. It also reduces appearance-motion ambiguity.

As STORM and DrivingRecon do not rely on LiDAR at test time (they used LiDAR as supervision during training same as Flux4D), we also conduct additional experiments to demonstrate that Flux4D can also operate in a LiDAR-free mode (while still processing Gaussians in 3D space) by using off-the-shelf monocular depth estimation model, namely Metrics3D [5]. Specifically, we removed LiDAR points entirely and instead voxelized the projected monocular depth with a 5 cm resolution, retaining up to 2M points, and then trained the model following the same 3D processing pipeline.

We observed that the flow estimation performance remains comparable, and in some cases, the visual realism improves in background regions (e.g., buildings) due to the broader coverage provided by monocular depth, particularly in areas where LiDAR sparsity limits reconstruction quality. Moreover, combining both LiDAR and Metrics3D depth yields the best overall realism.

Legend: _D = Dynamics only

Q3: Improvement from Iterative Refinement
A3: The iterative refinement module in Flux4D is designed primarily to enhance reconstruction and simulation fidelity and aim to bridge the gap with state-of-the-art per-scene optimization approaches like StreetGS and NeuRAD. It helps improve fine-grained visual details and reduce artifacts as shown in Tab. 5 and Fig. A2. However, the core motion reasoning and flow estimation are already well captured by the single forward-pass model, which is named Flux4D-base in the Main paper.

To illustrate this, we summarize the metrics for STORM, Flux4D-base, and the full Flux4D model below. As shown in the table, Flux4D-base already outperforms STORM significantly across all metrics, especially in visual quality of dynamic regions and flow estimation, while the full model with refinement further improves overall reconstruction fidelity (PSNR, SSIM).

Legend: D = Dynamics only

[1] Chodosh et al., Re-Evaluating LiDAR Scene Flow for Autonomous Driving, 2024.
[2] Khatri et al., I Can't Believe It's Not Scene Flow!, 2024.
[3] Charatan et al., pixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstruction, 2024.
[4] Xu et al., DepthSplat: Connecting Gaussian Splatting and Depth, 2025.
[5] Hu et al., Metric3d v2: A versatile monocular geometric foundation model for zero-shot metric depth and surface normal estimation, 2024.

We thank the reviewer for taking precious time in checking our responses. We hope our answers and additional results address your concerns well. Specifically,

Q1/A1: We compared DrivingRecon in Table 3 (main paper) and conducted additional comparisons with STORM as suggested.
Q2/A2: We clarified that Flux4D adopts a fundamentally different architecture that learns flow and appearance directly in 3D space with an approximate geometry initialization. This 3D design yields a more compact and geometrically consistent representation across views, improving scalability and enabling explicit multi-view flow reasoning. It also reduces appearance-motion ambiguity.
Q3/A3: We highlighted the benefits of the iterative refinement module, which further improves reconstruction fidelity beyond the already strong performance of the single forward-pass model (Flux4D-base).

Thanks again for the insightful review and suggestions. Please let us know if you have any additional or follow-up questions. We will be more than happy to address them. Any follow-up discussions are highly appreciated!

Thank you for the thoughtful review and valuable feedback. We are pleased that the paper was found to be “well‑written” and that the experiments and analysis are “sufficient.” We now address the concerns.

Q1: Scope beyond driving scenes: Inappropriate to use "4D" to name the method,evaluations on diverse 4D scenes
A1: Thank you for pointing this out. As stated in the paper (Abstract: L8–10; Introduction: L59–60; Method: L118–121), Flux4D is a simple and scalable framework for 4D reconstruction of large‑scale dynamic driving scenes. We agree that driving scenes primarily involve sparse rigid motions. To better clarify this scope, we will consider renaming the submission to “FluxDrive: Unsupervised 4D Reconstruction for Driving Scenes.”

We further note that our method handles short‑horizon non‑rigid motion (e.g., pedestrians and cyclists) on the Waymo Open Dataset and Argoverse 2, as shown in Fig. 7 thanks to the higher‑order polynomial motion models used in the final model (Sec. 3.4, Table 5, Appendix A.1). These motions and scenes are unseen during training, demonstrating the effectiveness and generalizability of our model on driving scenes. Regarding Eq. 2, it represents the motion model used in Flux4D-base, which can already achieve reasonable performance for outdoor driving scenes over short time horizons, as shown in our ablations. This observation aligns with existing works such as GVFi [1] and STORM [2].

Finally, as acknowledged in the method and limitation sections, highly dynamic, complex motions remain challenging (L290–291). Exploring more advanced velocity models or implicit flow representations is an exciting direction for future work (L188–189).

Q2: Can it generalize to unseen 4D motions or just unseen vehicle velocities?
A2: As discussed in Q1/A1, our method can generalize beyond vehicle velocities. In Fig. 7, we show that Flux4D can generalize to unseen non‑rigid motions (e.g., pedestrians and cyclists) on the Waymo Open Dataset and Argoverse 2. Moreover, as suggested by Reviewer QsPn, we conduct additional experiments to evaluate scene flow against test-time scene flow estimation methods (NSFP [3] and FastNSF[4]) using standard scene flow metrics, namely Threeway-EPE (EPE-3way) [5] and Bucketed Normalized EPE (Mean S, Mean D) [6].

As shown in the table, Flux4D achieves competitive performance across most metrics. Notably, although our approach is designed for unsupervised reconstruction and simulation rather than scene flow estimation, it outperforms other methods on challenging cases, including smaller or less common object categories and non‑rigid actors such as wheeled VRUs, other vehicles, and pedestrians, as shown in bucketed evaluations. This highlights the generalizability and effectiveness of large-scale unsupervised training.

Pandaset Table (1/2)

Pandaset Table (2/2)

Legend: BG = Background, CAR = Car, WVRU = Wheeled VRU, VEH = Other Vehicles, PED = Pedestrian, S = Static, D = Dynamics

Q3: The inference speed of the method is slower than existing approaches, which restricts its application in vehicle driving scenarios.
A3: Our method is designed for offline reconstruction and simulation, not real-time onboard perception. In this context, Flux4D offers significant advantages in both speed and reconstruction quality over state of the art, unveiling the possibility for scalable self-driving simulation.

First, Flux4D largely bridges the gap compared to state-of-the-art supervised self-driving reconstruction methods (as recognized by Reviewers QsPn, tD1B, Ew2X). It is over 10× faster than generalizable approaches like G3R while achieving higher-quality results. Compared to per-scene optimization methods such as StreetGS and NeuRAD, Flux4D offers up to 500× speedup without requiring any labels, making it significantly more scalable for large-scale simulation.

Second, although system-level efficiency was not the primary focus of our submission, we analyzed the inference pipeline and removed redundant operations, improving our reconstruction time to 1.8 s. If the iterative refinement module is disabled, our model becomes Flux4D-base in the main paper, which reconstructs the scene in 0.13 s and still achieves significant visual quality improvements over existing unsupervised feed-forward methods (DepthSplat, L4GM, STORM) on dynamic actors as shown below. These results are obtained using a single RTX A5000 GPU. Running on a more advanced GPU such as an H100 would also provide a substantial speedup. Additionally, inference acceleration via TensorRT or quantization is straightforward and expected to yield further gains.

Legend: _D = Dynamics only

Third, while some existing feed-forward methods offer faster inference, they typically operate at significantly lower resolutions (e.g., $160 \times 240$ for STORM) and/or suffer from limited fidelity with noticeable artifacts (see Fig. 3). This compromises downstream usability and leads to a larger domain gap. Moreover, these approaches either do not support key simulation capabilities (e.g., actor removal, insertion) or produce significant artifacts, due to insufficient reconstruction quality and scene decomposition.

In conclusion, we reiterate that Flux4D is designed for offline driving-scene simulation. Unlike traditional pipelines that require manual annotations and hours of optimization, Flux4D offers an order-of-magnitude speedup with competitive fidelity in a fully unsupervised manner, making it well-suited for scalable, simulation-driven autonomy development. We will include updated timing results and discussion in the revision.

[1] Li et al., GVFi: Learning 3D Gaussian Velocity Fields from Dynamic 360 Videos, 2025.
[2] Yang et al., STORM: Spatio‑Temporal Reconstruction Model for Large‑scale Outdoor Scenes, 2025.
[3] Li et al., Neural Scene Flow Prior, 2021.
[4] Li et al., Fast Neural Scene Flow, 2023.
[5] Chodosh et al., Re-Evaluating LiDAR Scene Flow for Autonomous Driving, 2024.
[6] Khatri et al., I Can't Believe It's Not Scene Flow!, 2024.

Thank you for the thoughtful review and valuable feedback. We are pleased that the method was recognized for its “simple” design, “ability to scale with data,” and “impressive” results. As suggested, we conducted additional experiments comparing with scene flow methods using standard scene flow metrics, supported interactive 3D visualizations, and evaluated Flux4D without LiDAR (using monocular depth estimation). We will include the results and analysis in the revision.

Q1: Evaluations on scene flow
A1: Thank you for the suggestion. While Flux4D primarily focuses on reconstruction and is not specifically designed for scene flow estimation, we agree that evaluating its performance compared with scene flow estimation methods using standard scene flow metrics is valuable. To this end, we conducted additional experiments comparing Flux4D with NSFP [1] and FastNSF [2] on PandaSet and Waymo Open Dataset (WOD). We also include comparisons with STORM [3].

As existing scene flow estimation methods cannot directly predict flows at novel timesteps, we evaluate scene flow on the input frames, following STORM. We restrict evaluation to LiDAR points within the camera field of view (FoV) and report standard metrics including EPE3D, Acc@5, Acc@10, angular error ($\theta$ error), Threeway EPE (BS: Background-Static, FS: Foreground-Static, FD: Foreground-Dynamics) [4], bucketed normalized EPE [5], and inference speed. For bucketed normalized EPE on WOD, as the provided semantic labels are not rich, we follow EulerFlow [6] and report the metrics on Background (with Signs), Vehicles, Pedestrian, and Cyclists only.

Although not specifically designed for scene flow estimation, Flux4D achieves superior performance across most scene flow metrics using only reconstruction-based supervision (RGB + depth). Notably, it outperforms other methods on smaller or less common object categories such as wheeled VRUs, other vehicles, and pedestrians, as shown in bucketed evaluations. We will include these results, visualizations, and further analysis in the revised version.

Pandaset Table (1/2)

Pandaset Table (2/2)

Legend: BG = Background, CAR = Car, WVRU = Wheeled VRU, VEH = Other Vehicles, PED = Pedestrian, S = Static, D = Dynamics

WOD Table (1/2)

WOD Table (2/2)

Legend: BG = Background, VEH = Vehicles, PED = Pedestrian, CYC = Cyclists, S = Static, D = Dynamics

Q2: 3D visualizations in the appendix (or better an interactive website)?
A2: Thank you for the suggestion. We now support interactive 3D visualizations, including 3D point, camera and flow visualization using the viser library, as recommended. Due to NeurIPS rebuttal guidelines prohibiting any links, we are unable to share the interactive visualizations at this stage. However, we will include representative 3D visualizations in the camera-ready version and provide interactive examples on the project website.

Q3: Velocity reweighting: Is this using the flow predictions? What happens at the start of training when presumably the velocity estimates are not good?
A3: Yes, we use the predicted flow to reweight the photometric loss and enhance the supervision for dynamic regions. As detailed in Supplementary Sec. A.1 (Eq. 1, Line 16-20), we apply a pixel-wise reweighting based on the rendered velocity magnitude:

$$\mathcal{L} = (1 + \left\Vert \text{sg}(\mathbf{v_r}) \right\Vert) \cdot \mathcal{L}_{\text{rgb}},$$

where $\text{sg}(\cdot)$ denotes the stop-gradient operation and $\mathbf{v_r}$ is the rendered velocity in image space. For stability, we clip $\left\Vert\mathbf{v_r}\right\Vert$ to $[0, 10]$. At the start of training, the predicted velocity magnitudes are close to zero, so all pixels are weighted equally. Even when early velocity estimates are inaccurate, the clipping and base weighting ensure stable gradients and effective learning throughout training.

Q4: Flux4D without LiDAR (e.g., monocular depth estimation)
A4: Thank you for pointing this out. Although LiDAR is an important component of Flux4D, it is not strictly necessary. In our framework, LiDAR primarily serves as an initialization for the 3D scene structure and does not need to be highly accurate. To evaluate performance without LiDAR, we conducted an additional experiment using monocular depth estimated by off-the-shelf Metric3D [7] model, as suggested. Specifically, we removed LiDAR points entirely and instead voxelized the projected monocular depth with a 5 cm resolution, retaining up to 2M points, and then trained the model following the same 3D processing pipeline. The results are shown below.

We observed that the flow estimation performance remains comparable, and in some cases, the visual realism improves in background regions (e.g., buildings) due to the broader coverage provided by monocular depth, particularly in areas where LiDAR sparsity limits reconstruction quality. Moreover, combining both LiDAR and Metrics3D depth yields the best overall realism.

We will include the quantitative results, visualizations, and further analysis in the revision.

Legend: _D = Dynamics only

[1] Li et al., Neural Scene Flow Prior, 2021.
[2] Li et al., Fast Neural Scene Flow, 2023.
[3] Yang et al., STORM: Spatio-Temporal Reconstruction Model for Large-Scale Outdoor Scenes, 2025.
[4] Chodosh et al., Re-Evaluating LiDAR Scene Flow for Autonomous Driving, 2024.
[5] Khatri et al., I Can't Believe It's Not Scene Flow!, 2024.
[6] Vedder et al., Neural Eulerian Scene Flow Fields, 2025.
[7] Hu et al., Metric3d v2: A versatile monocular geometric foundation model for zero-shot metric depth and surface normal estimation, 2024.

Thank you for the thoughtful suggestions and follow-up discussion. We are glad to hear that the additional evaluations helped contextualize the results. We agree that unifying SOTA flow and SOTA reconstruction is an exciting direction for future work, with the potential to benefit both tasks and enhance generalization. We hope that Flux4D takes an important step toward building this connection!

Thanks again for the insightful review and suggestions! Please let us know if you have any further comments or questions.