NFL-BA: Near-Field Light Bundle Adjustment for SLAM in Dynamic Lighting

Lighting Model Limitations

We agree with the reviewer, and we clarified in the paper that the NFL-BA formulation only models diffuse reflectance and direct illumination, ignoring specular reflectance, subsurface scattering and inter-reflections. While these effects are important, physically modeling them in an optimization framework is significantly challenging and an open problem in the community. This is because these effects introduce non-differentiable elements (e.g., path-tracing for inter-reflections), highly non-convex functions (e.g., sub-surface scattering and specular reflections modelled as Spatially Varying BRDF), and additional parameters for material modeling.

During experimentation on real data with specular reflections (both in endoscopy and in-the-wild captures), we first detect and exclude specular regions via a simple intensity mask. We convert each frame to grayscale and discard pixels above a 0.9 intensity threshold, filtering out bright highlights. While this approach handles most specular artifacts, it cannot correct for reflective surfaces beyond the mask. The C3VD phantom dataset and self-captured in-the-wild dataset have predominantly matte surfaces, so this works well. However, as noted, the Colon10K clinical endoscopy dataset contains significantly more specular highlights, leading to moderate degradation in reconstruction. We believe future research is required to explicitly account for specular reflections in the presence of dynamic lighting.

Angular Attenuation

We appreciate the reviewer’s point that setting ignoring angular attenuation (β=0) may overestimate irradiance under wide-angle lighting such as smartphone flashlights. While β=0 is physically justified for tightly collimated endoscopic LEDs, we validated this choice on two self-captured indoor sequences with varying β.

Table A: Ablation of the angular-attenuation coefficient on two self-captured indoor sequences, reporting translational ATE and LPIPS across β ∈ {0.00, 0.25, 0.50, 0.75, 1.00}.

We found that β=0 provides competitive mean accuracy with the low variance, confirming our default choice. Non-zero β values can yield marginal improvements in ATE or LPIPS, but variability increases and our coarse search did not pinpoint a universally optimal value.

Given the minimal gains versus added complexity of tuning β, we retain β=0 in our current formulation. In future work, we intend to explore learning β as a per-scene or per-dataset parameter to enable fine-grained angular attenuation without extensive manual search.

Experiment Robustness

We have included updated tables with averages and standard deviations for 5 runs per trial. Due to lack of time and resources during the rebuttal period, we only included evaluations for EndoGSLAM on C3VD, and MonoGS on Self-Capture. We will include all 5 trials, for all experiments, in the final version.

Table 1. Quantitative Evaluation on the C3VD under dynamic lighting, comparing EndoGSLAM with standard Photo-BA versus NFL-BA.

Table 2: Quantitative results on four self-captured indoor scenes under dynamic lighting, comparing MonoGS with standard Photo-BA versus NFL-BA

Full MonoGS Evaluation

As suggested, we evaluated MonoGS with RGB-only input and compiled all MonoGS experiments below. Overall, NFL-BA yields substantial reductions in both translational and rotational errors across all depth initialization modes. Additionally, map quality consistently improves, demonstrating that our lighting-aware loss provides complementary information to geometric depth cues, enhancing SLAM robustness even when low-quality depth is available.

Table 3: Quantitative results on the C3VD dataset with 4 depth inputs including RGB-only. Replacing Photometric BA with NFL-BA significantly improves tracking and mapping regardless of input depth guidance.

Impact of NFL-BA on EndoGSLAM

EndoGSLAM is a highly specialized SLAM system tailored for the C3VD dataset through a set of carefully engineered priors, such as constant-velocity motion assumptions and brightness-dependent masking, that together deliver outstanding performance in a very specific setting.

Dataset specialization: EndoGSLAM is heavily tuned for C3VD and struggles with out‑of‑domain data such as Colon10K (Figure 7), where motion is less uniform and specular reflections are more prevalent. For both test sequences, EndoGSLAM completely fails to produce a continuous structure due to the absence of ground truth depth and broken assumptions

Engineered for Ground Truth depth: When ground‑truth depth is available, EndoGSLAM already performs extremely well, leaving minimal room for improvement. When the depths are perfectly aligned, NFL‑BA’s shading‑based optimization contributes little and may occasionally conflict with the built-in priors. However, under noisy depths, EndoGSLAM performance drops sharply, and NFL‑BA provides significant gains by leveraging physically‑based lighting cues to stabilize pose and reconstruction.

In short, the small degradation under GT depth reflects EndoGSLAM’s rigidity and high baseline rather than a fundamental weakness of NFL‑BA. In typical clinical scenarios, NFL‑BA offers a clear benefit.

Perceived Simplicity of NFL‑BA

Although NFL-BA is conceptually straightforward, its novelty is in embedding a physically grounded near-field lighting model directly into a differentiable bundle-adjustment loss. To our knowledge, no prior Bundle Adjustment (BA) formulation explicitly accounts for dynamic, co-located illumination. This reformulation not only captures essential lighting physics but also remains stable during optimization. Moreover, the method’s minimal implementation overhead makes it a drop-in enhancement for a wide range of neural SLAM systems without altering their core architectures. Despite its simplicity, NFL-BA delivers state-of-the-art results, reducing MonoGS’s absolute trajectory error by 37% and EndoGS’s by 14% on the challenging C3VD colonoscopy dataset.

Code Availability

We understand the importance of reproducibility and will release the full codebase upon acceptance.

Dear reviewer,

Thank you again for your thoughtful and detailed feedback. If you have any further questions or concerns regarding our rebuttal or the paper, we'd be happy to clarify or provide additional context.

Choice of Hyperparameters

We appreciate the reviewer’s concern and would like to clarify that we did not tune the hyperparameters {λ_ren, λ_geo} individually for different SLAM frameworks. Instead, we adopted the same values used by the respective base methods. Neural SLAM systems such as MonoGS, EndoGS, and NICE-SLAM inherently use different hyperparameter settings for photometric loss (λ_ren​) and geometric loss (λ_geo​), both in the tracking and mapping stages, due to differences in their underlying 3D scene representations and optimization strategies (e.g., keyframe selection, voxel grid resolution).

Importantly, we did not modify or tune these values for different datasets either. The same set of hyperparameters was used consistently across all evaluation sequences and datasets—including C3VD and our in-the-wild captures. This design choice reflects our goal of making NFL-BA a plug-and-play bundle adjustment loss, compatible with existing neural SLAM pipelines without requiring any additional tuning. By relying on the default settings of each framework, we demonstrate that NFL-BA can be seamlessly integrated and consistently improve performance across diverse settings.

NFL-BA on scenes captured with static lighting (TUM RGB-B, Replica)

Near-Field Lighting Effects occur when a light source moves very close to the surface. In endoscopy, the light source and camera move together within millimeters of the tissue, producing a drastic inverse-square fall-off, deep shadows, and specular highlights. These dynamic lighting phenomena are more extreme in endoscopy than in typical settings. General indoor SLAM datasets (TUM RGB-D, Replica) assume far-field or static lighting and thus do not exhibit the dynamic illumination patterns that NFL-BA is designed to correct. Hence, applying NFL-BA on these datasets will not make any sense.

Note that NFL-BA is designed as a plug-and-play Bundle Adjustment loss for any Neural SLAM algorithm, both NeRF & 3DGS-based. This means for any SLAM pipeline, we can easily replace Photometric Bundle Adjustment designed for static lighting with NFL-BA designed for dynamic near-field lighting, without any need to change the SLAM framework.

Motivation for using the colonoscopy dataset for benchmarking

Endoscopy provides a highly challenging near-field lighting scenario, making it an ideal testbed to demonstrate NFL-BA’s efficacy. Endoscopy is a minimally invasive imaging technique where a slender and flexible tube equipped with a light and camera is used to inspect internal organs and tissues of the body via an opening, e.g. mouth or anus. Endoscopy is routinely used for cancer screening, e.g. in colons and lungs; for identifying potential issues in the digestive tract, e.g. ulcers, polyps, hemorrhoids; and respiratory problems, e.g. infections, an abnormal narrowing of the airway, and collapsed lungs, and many others. Robust 3D reconstruction and camera tracking from endoscopy enable various downstream applications: (i) improving 3D visualization during an endoscopy procedure and providing guidance to unsurveyed regions, (ii) automatic quantification of various geometric measures of airway or colon to assist in abnormality detection, and (iii) enabling assisted or automatic navigation of the endoscope without tissue damage. Since high-quality publicly available endoscopy datasets for the airway are unavailable, we mainly focused on colonoscopy, since C3VD provides high-quality data with ground-truth.

Evaluation of Center-Crop Baseline

We thank the reviewer for suggesting the center-crop baseline. We compared MonoGS optimized on only the central 75% and 50% of each frame against full-frame MonoGS + NFL-BA on two sequences, measuring translational ATE (ATE_T), Chamfer distance (CD), and reconstructed point count.

Table A: Center-cropping does improve trajectory error upon the original baseline, but suffers from map sparsity, while full-frame NFL-BA maintains both accuracy and dense reconstructions.

Although center-cropping improves camera tracking performance (ATE_t) of MonoGS and gets close to NFL-BA, it results in significantly worse reconstruction in terms of quality (CD) and density (number of points). This is because for camera localization, not all the pixels are essential, and focusing only on central pixels in the image can eliminate the effects of near-field lighting. In contrast, for reconstruction, all pixels in the image matter. This shows the necessity for developing a principled mechanism for handling dynamic near-field lighting, as presented by our work NFL-BA.

NFL-BA Performance Variation with Length of Test Sequence

We thank the reviewer for suggesting a thorough evaluation on extended trajectories. C3VD's Seq2 spans over 4,000 frames with only ground-truth poses, allowing us to measure cumulative drift over time. We note that Seq2 is a screening video that was not utilized in the original paper.

Table B: We evaluate ATE_T (cm) on C3VD Sequence 2 at varying lengths. NFL-BA matches or beats Photo-BA on short sequences and demonstrates long-term stability.

We observe that NFL-BA matches Photometric BA on short sequences but increasingly outperforms it as trajectory length grows. As a plug-and-play bundle-adjustment loss, NFL-BA inherits the stability of the underlying SLAM system while further enhancing long-term robustness under dynamic near-field lighting.

Performance of NFL-BA with Ground-truth Depth

We acknowledge that NFL-BA’s improvement over existing methods is modest when ground truth depth is available, as reported in Table 1 of the main paper. When ground truth depth is available, the optimization objective becomes easier. Hence, inaccurate modeling of photometric BA loss matters less. In comparison, when ground truth depth is not present, the optimization is challenging and inaccurate photometric modeling leads to larger error.

In the case of EndoGSLAM, it is a highly specialized endoscopic SLAM system with multiple engineered priors, including constant‑velocity motion assumptions and brightness‑dependent masking. Unlike methods that rely primarily on photometric cues, EndoGSLAM leverages a diverse set of cues and domain‑specific heuristics. When ground‑truth depth is available, EndoGSLAM already performs extremely well, leaving minimal room for improvement. When the depths are perfectly aligned, NFL‑BA’s shading‑based optimization contributes little and may occasionally conflict with the built-in priors. However, it is heavily tuned for C3VD and struggles with out‑of‑domain data such as Colon10K (Figure 7), where motion is less uniform and specular reflections are more prevalent.

Additionally, we would like to address the inconsistencies with translational and rotational ATE. Near-field lighting produces large changes in pixel intensity for small shifts in camera position, making the loss highly sensitive to translational errors. By contrast, rotational misalignments produce subtler shading differences. As a result, our loss yields larger relative gains on translation than on rotation.

Evaluation Without Depth Initialization

To quantify how NFL-BA contributes under varying depth guidance, we evaluated MonoGS on the C3VD dataset with four depth initialization modes: no depth (RGB-only), DaHybrid, PPSNet, and Oracle. We compared Photometric BA against NFL-BA and reported both tracking and reconstruction results.

Table C: Quantitative results on the C3VD dataset with 4 depth inputs including RGB-only. Replacing Photometric BA with NFL-BA significantly improves tracking and mapping regardless of input depth guidance.

Overall, NFL-BA yields substantial reductions in both translational and rotational errors across all depth initialization modes. Additionally, map quality consistently improves, demonstrating that our lighting-aware loss provides complementary information to geometric depth cues, enhancing SLAM robustness even when low-quality depth is available.

Minor Points

Figure 7 contains results for the colon10k dataset, which contains endoscopy videos captured on real patients in clinical setting, which has no ground truth annotations. To see the source videos or point clouds for the MonoGS results, please check out our supplementary submission site. Additionally, we will ensure that all SOTA labels are complete in Table 2 in the final submission.

Dear reviewer,

Thank you again for your thoughtful and detailed feedback. If you have any further questions or concerns regarding our rebuttal or the paper, we'd be happy to clarify or provide additional context.