AG-SLAM: Active Gaussian Splatting SLAM

It is unclear how much overhead there is to compute the fisher information. Eq. 16 requires the Jacobian wrt to the rendering output, does this mean that the Jacobian is (num_pixels)x6?

Though it's a SLAM system, run time is not reported which makes it hard to assess whether the approach is practical.

The PSNR evaluation is somewhat unfair. MonoGS directly minimizes PSNR, whereas other SLAM methods optimize different objectives. For a fairer comparison, all methods should perform pose refinement using MonoGS, with the SLAM trajectory as initialization.

1. Limited Contribution: While I appreciate the author’s efforts in integrating 3D Gaussian Splatting (3DGS) with active SLAM, I think the technical contribution presented in this paper to be somewhat limited and may not be insufficient for publication at the ICLR level. The core methodology lacks substantial innovation, as the primary technology and approaches used do not introduce significant advancements beyond existing work. In my opinion, this paper appears to utilize existing methods for calculating the Hessian matrix and EIG, combining them with the $\log\det(\cdot)$ of the Fisher Information matrix to create a balanced objective function for exploration and improved self-localization. This approach is a relatively common technique in active SLAM and active localization fields. The challenging part of gradient calculation of the 3DGS rendering model with respect to the pose, $\frac{\mathcal{D} f(\mathrm{x})}{\mathcal{D} \mathbf{x}}$, also relies on previously established methods. As such, the novelty and impact of the proposed approach feel somewhat limited in scope.

2. Unclear Definitions: The paper could benefit from clearer definitions of key terms and variables to improve readability. For example, the authors mention minimizing the difference between the image observation $\mathbf{y}$ and the 3DGS parameters $\mathbf{w}$ through the rendering model $f(\mathbf{x}, \mathbf{w})$, but they do not clarify detailed information of these variables such dimensionality. Another example is the function $g(\cdot)$ in Equation (5), which lacks a clear definition, making it unclear how this function is formulated or what role it plays within the proposed framework. Including additional details on variable dimensions and implementation specifics would significantly aid readers in understanding the methodology and reproducibility of the work.

3. Computational Cost: Although the authors claim that their approach functions as an online SLAM method, there is limited discussion on the computational efficiency or performance overhead of the proposed approach. Since online SLAM typically requires low latency and real-time processing capabilities, an experimental evaluation of computational efficiency would be beneficial. Including such results would strengthen the paper by validating its practical applicability and providing a more comprehensive view of its performance.

• The method description needs significant improvements for clarity of notation and explanations. For instance, it lacks proper introduction and definition of the expected information gain that is optimized.
• The proposed calculation of I(w) does not make sense as it should measure the entropy of the 3DGS map parameters after initialization with the previous training views before integrating further view information. It is not clear why sampling other views than the training views should improve information about this initial parameter estimate.
• It is not clear what the quantity in eq. 13 is and why it should be considered. For a mutual information, the conditional entropy should be subtracted. It seems what the authors would like to achieve is to compute the expected information gain along a path. For this a simulation of the measurements along the path and incremental integration of measurements into the 3DGS would be required though to be able to measure the conditional entropy given the sequence of measurements.
• Representing poses as Lie algebra elements at identity (i.e., axis-angle vectors for the rotation part) is generally suboptimal for pose optimization and covariance representation as the rotation representation is not unique and has singularities at multiples of 2\pi.
• l. 503, what quantifies a "main area of the scene" ?
• Restricting the trajectory length to 2000 steps seems artificial. Rather, a significantly larger number steps should be used as time out and the approaches should be given chance to explore the whole environment. Which method explores the scene faster? Is there a trade off between exploration time, localization and reconstruction accuracy ? Do all methods explore the environment fully?
• How are hyperparameters of the proposed approach and the baselines obtained ? Is the comparison fair?
• Why are the specific HM3D scenes chosen for evaluation? Have parameters of the methods been tuned on different scenes?

Minor comments:

• "so we set the pose estimate of the MonoGS backend to the one from ANS" - this should be phrased the other way round.
• please use booktabs standard style for format tables.

1. The writing is often clumsy. You should proofread your document carefully.
2. The good mapping and exploration performance exhibited by the proposed method may depend heavily on the utilization of the 3DGS, but the methodology section rarely deals with the description of the 3DGS integration. Following the current description, it seems that the proposed active pose selection method can be applied to any 3D representation, not just 3DGS.