GS-Hider: Hiding Messages into 3D Gaussian Splatting

Thank you for your constructive comments! If there are any additional comments to be added, please continue the discussion with us.

$\textcolor{red}{**The supplementary rebuttal PDF file can be found at the bottom of the overall response**}$.

Weakness #1, Question #1 and Question #2: Additional computational overhead.

• We kindly remind the reviewer that we have already provided the rendering time of each scene in our $\textcolor{red}{**Table 2**}$ of the main paper and presented the FPS of our GS-Hider in $\textcolor{red}{**Line 236**}$. It achieves 45 FPS rendering speed, which surpasses the real-time rendering requirement of 30 FPS. Meanwhile, the added inference time of scene decoder (5 Conv layers) is only $\textcolor{red}{**0.006s**}$, only accounts for $\textcolor{red}{**0.006/0.0222=27**}$% of the total rendering time.

• The rendering speed of GS-Hider can be further improved by adjusting some hyperparameters, such as reducing the number of convolutions in the scene decoder ($N$) and the dimension of feature attributes ($M$), sequentially pruning Gaussian points in ascending order of Gaussian’s opacity. We report the FPS of GS-Hider under different settings and compare it with the original 3DGS on the $\textcolor{red}{**Table 1**}$ of the rebuttal PDF. The FPS of our lightest GS-Hider ($M=8, N=5$) reaches $\textcolor{red}{**71.429**}$ and is comparable to the original 3DGS with an acceptable rendering quality.

Weakness #2 and Question #3: Prove the hidden scene is not memorized by the decoder.

• We have conducted experiments of hiding two hidden scenes, as detailed in $\textcolor{red}{**Section 4.6**}$ of the main paper. This proves that our decoder is not storing or memorizing the secret information.

• Our message decoder is very lightweight with only $\textcolor{red}{**5**}$ convolution layers. It only contains $\textcolor{red}{**0.465 M**}$ parameters, which is far from enough to memorize complex 3D scenes. In fact, the geometrical and structural information is mainly embedded in the coupled feature, and the role of the decoder is merely to $\textcolor{red}{**extract and decouple the secret information**}$, not to memorize the scene watermark.

• To show the role of our decoder, we further input the rendered coupled feature from another scene like 'playroom' to the message decoder that is trained to hide the scene 'bicycle'. The results are presented in $\textcolor{red}{**Figure 4**}$ of the rebuttal PDF. We find that the rendered scene retains most of the geometric structure of the 'playroom' scene, with only some colors resembling those of the 'bicycle' scene. This indicates that our decoder itself does not have the capability to memorize secret information.

Weakness #3: A lack of theoretical justification for why the coupled feature works.

• The reason of the coupled feature representation works so well is that the feature attribute $\boldsymbol{f}_i$ has sufficient capacity and high flexibility. Combined with our decoder and designed loss function, it can effectively fuse two scenes, and hide secret information in some visually insensitive areas and the redundant feature channels. More analysis can be found in our response to the Question #1 of reviewer X8jQ.
• Considering the black-box nature of deep learning, thoroughly analyzing the process of embedding information into high-dimensional features, and providing theoretical proof is an open, challenging, and interesting issue. We appreciate your valuable suggestions and will leave this for future work.

Weakness #4: Comparison to recent work.

According to your valuable suggestion, we have tried our best to migrate the pipeline and decoding network of StegaNerf [1] to the 3DGS steganography task. Specifically, we feed the output of 3DGS to the decoding network of StegaNerf and let it approximate the hidden 3D scene. The results are reported on $\textcolor{red}{**Table 2**}$ of the rebuttal PDF. We find that GS-Hider is much better than 3DGS+StegaNeRF in the reconstruction quality of hidden scenes. If you find other comparison methods suitable for 3DGS steganography, please let us know and we will compare them accordingly.

Weakness #5: Generalization to other datasets and hidden information.

• Different dataset: We have supplemented the experiments on two datasets, namely Tank&Template and Deep Blending, on $\textcolor{red}{**Table 3**}$ and $\textcolor{red}{**Figure 5**}$ of the rebuttal PDF. For Tank&Template, we hide the scene 'bicycle' into the two original scenes. For Deep Blending, we insert and hide the 'lego' and 'hotdog' in the Nerf synthetic dataset to the original scenes. It can be found that our GS-Hider can still achieve good results on these two datasets.

• Different type: Our GS-Hider can support hiding scenes or a copyright image in our main paper. It can also be applied to hide bits or audio since they can also be treated as a special image. We will realize it in our future work.

Limitations and broader impacts

• We have reorganized the limitations of our approach and potential improvements in depth in our response to Weakness #1 of reviewer koxG.

• Our GS-Hider needs to be used in conjunction with an online copyright protection platform to record the exclusive copyrights of different users and scenarios to prevent copyright conflicts. Meanwhile, GS-Hider can be combined with key technology for further protection to prevent users from abusing our models and sensitive data.

Reference

[1] Steganerf: Embedding invisible information within neural radiance fields, in ICCV 2023.

Thank you for your response. It addresses my concerns regarding potential added overhead of inference time. So I raise the rating. I recommend that the revision could include the supplied results in your rebuttal.

Thank you for your valuable comments! If there are any additional comments to be added, please continue the discussion with us.

Weakness #1: Minor Limitations.

Due to limited space, we only discussed the limitations of our method in terms of rendering quality and speed in the supplementary material. We will delve deeper into these two aspects, provide additional insights, and introduce our potential improvements. Our limitations are reorganized as follows.

• Compromised rendering quality: Since the feature attribute $\boldsymbol{f}_i$ does not consider view-dependency compared to spherical harmonics, and we need to hide the secret scene while representing the original scene, our rendering quality is somewhat inferior to the original 3DGS. In fact, we inevitably need to make a trade-off between rendering quality and steganography capacity. However, our GS-Hider is a universal framework that can be integrated with the latest 3DGS variants, such as Mip-splatting [1], to enhance rendering performance. Details can be found in the response to Weakness #2 of reviewer S9BH. Meanwhile, the current designs of the scene and message decoder are relatively simple. Integrating more efficient neural rendering and decoding designs (such as Scaffold-GS [2]) can also help improve the overall rendering quality of the framework.
• Decreased rendering speed: Due to the rasterization of high-dimensional features and the use of network decoding, although we can still achieve real-time rendering, the rendering speed has decreased compared to the original 3DGS. However, we can easily improve rendering speed by pruning Gaussian points, reducing the dimension of feature attributes $\boldsymbol{f}_i$, and decreasing the number of convolution layers or feature kernels. As shown in $\textcolor{red}{**Table 1**}$ of the rebuttal PDF, these approaches do not significantly impact rendering quality.

Question #1: How to resist Eve's re-rendering?

Thank you for presenting such an interesting scenario. If Eve re-renders a GS using Alice's trained model, our copyright protection still holds for the following reasons:

• First, if Eve re-renders a GS, he cannot decode any pre-embedded copyright image or secret scene from it, making Eve's GS unauthorized. Therefore, Eve cannot prove that he owns the copyright of this GS, and the ownership of this GS model still belongs to Alice.
• Second, our GS-Hider works in conjunction with an online copyright database. When Alice uploads her model, the copyright image is registered in the database. If a similar GS scene is encountered later, the decoded copyright image must be matched with the one in the database. Otherwise, it will be judged as infringement.
• Third, the cost of re-training a GS is high. Eve would need to spend almost the same computational resources as Alice to steal such a 3D scene, which is difficult for the average thief to achieve.

Reference

[1] Mip-Splatting: Alias-free 3D gaussian splatting, in CVPR 2024.

[2] Scaffold-gs: Structured 3d gaussians for view-adaptive rendering, in CVPR 2024.

Thank you for your constructive comments! We hope that our response will address all of your concerns. All discussions and supplementary analyses will be included in our revised version. If there are any additional comments to be added, please continue the discussion with us.

$\textcolor{red}{**The supplementary rebuttal PDF file can be found at the bottom of the overall response**}$.

Weakness #1: Compromised rendering quality and speed.

• First, our rendering quality and speed are still $\textcolor{red}{**acceptable**}$.

  • In the case of hiding a 3D scene, our rendering quality only drops a little bit (about 1dB) and is comparable to the original 3DGS and other 3D rendering methods (such as Instant NGP).
  • Meanwhile, our rendering speed can reach 45 FPS, $\textcolor{red}{**which far exceeds the real-time rendering requirement of 30 FPS**}$.

• Second, our method has $\textcolor{red}{**unique advantages**}$ compared with the original 3DGS.

  • As shown in Table 1 of the main paper, our GS-Hider significantly reduces the storage space by $\textcolor{red}{**385.05MB**}$ compared to the original 3DGS.
  • Our GS-Hider has better privacy and can hide a 3D scene or a copyright image with high quality, which is suitable for tasks such as encrypted transmission and copyright protection.

• Third, the rendering speed of our GS-Hider can be $\textcolor{red}{**easily improved**}$.

  • We can apply GS compression methods to reduce the number of Gaussian points and improve rendering speed. As shown in Table 2 of the main paper, sequentially pruning Gaussian points by 25% hardly affects the rendering quality and can bring a 20% improvement in rendering speed.
  • We can improve rendering speed by reducing the number of convolutions in the scene decoder or using a more efficient and lightweight network design. More FPS results of GS-Hider under different settings can be found on $\textcolor{red}{**Table 1**}$ of the rebuttal PDF.

Question #1: Where and how GS-Hider hides the secret information?

• First, the hidden scene information is concealed in the spatial high-frequency details of the coupled feature and some visually insensitive areas (such as artifacts, noise, and edges). The invisible hidden information in the coupled feature map will be amplified and decoupled by the message decoder, eventually forming an RGB hidden scene. We visualize the intermediate feature of the message decoder in the $\textcolor{red}{**Figure 1**}$ of the rebuttal PDF to illustrate this process.

• Second, the secret information is hidden in some redundant feature channels of the coupled feature field $\mathbf{F}\_{coup}$. To prove this, we randomly set some channels in $\mathbf{F}\_{coup}$ to $\mathbf{0}$, and eventually find that the hidden decoder can not reconstruct the complete secret scene. The results are presented in $\textcolor{red}{**Figure 2**}$ of the rebuttal PDF. This indicates that multiple feature channels are coupled and interact with each other, collectively storing the hidden information.

Thank you for your constructive comments! We hope that our responses can address your concerns. If there are still aspects that need further clarification, please feel free to continue the discussion with us!

Weakness #1: Limited accessibility and usability.

• Our method is actually very simple, efficient, and user-friendly. It does not require substantial computational resources than the original 3DGS. In fact, our storage space is even smaller than the original 3DGS because we use a more compact feature representation.
• Additionally, our method can be optimized end-to-end in a $\textcolor{red}{**single GTX 3090 Ti Server**}$, with training and decoding easily encapsulated into an interface. In the training phase, users only need to input the original scene and the hidden scene or a copyright image to render a secure and private 3DGS for publishing or sharing. In the verification phase, users only need to use a private message decoder to extract encrypted information from the Gaussian point cloud. This makes it accessible for users without a background in deep learning to use it effortlessly.

Weakness #2: Extension to other variants of 3DGS.

• According to your valuable suggestions, we realize a variant of GS-Hider based on Mip-splatting. Specifically, we retain the 3D smooth filter and 2D mip filter from Mip-splatting, only replacing the color attributes to high-dimensional features $\boldsymbol{f}_i$ to fit the GS-Hider framework. Then, we conducted experiments on 3D scene hiding on the mipnerf-360 dataset. The results are reported as follows. We also present some visualization results in $\textcolor{red}{**Figure 3**}$ of the rebuttal PDF file. This demonstrates that our GS-Hider is a universal steganography framework, not limited to specific 3DGS methods.

  Method	$\text{PSNR}_S$	$\text{SSIM}_S$	$\text{LPIPS}_S$	$\text{PSNR}_M$	$\text{SSIM}_M$	$\text{LPIPS}_M$
  Mip-Splatting	27.79	0.83	0.20	-	-	-
  Mip-GSHider (Ours)	26.25	0.79	0.24	25.26	0.76	0.34