MiraGe: Editable 2D Images using Gaussian Splatting

We appreciate the Reviewer’s feedback. We also thank Reviewer for appreciating the concept and work: “utilizing two opposite cameras to fit 3D Gaussians to represent 2D images is an interesting idea, and evaluation also shows the effectiveness of this idea”. Since the section for questions directed to us is empty, we have opted to refer to other sketches. We have made every effort to resolve any ambiguities. We hope the additional clarifications provide further clarity. We remain open and keen to provide further explanations if needed.

EDOA: The comparisons with DragGAN and PhysGen are not sound to me...

We would like to mention that representing a 2D image using Gaussian splatting for editing is a new concept, and it is difficult to compare it with existing tools. Therefore, we apply comparisons with existing solutions such as DragGAN and PhysGen, which shows the potential of our approach.

In the case of PhysGen, we chose these models to highlight their differences. An interesting case is PhysGen's input control, which offers limited editing capabilities. In PhysGen, the authors utilized a physics engine for training, with a diffusion model serving as an intermediary. However, this approach restricts user control. In contrast, we demonstrate that MiraGe allows direct use of a physics engine for Gaussian representation, eliminating the need for a complex diffusion model conditioned on the physics engine. Additionally, our representation is compatible with multiple engines and tools, including Taichi Elements and Blender. In summary our representation enables editing directly in 3D, making a comparison with simple pixel modifications insufficient. Ultimately, we use this comparison to highlight the strengths of our model and its potential applications.

C&E/ W

In 2D-MiraGe primitives are constrained to a 2D plane, meaning edits should only be made on a plane parallel to the selected one. Otherwise, during rotation, they will fade away, - much like a thin piece of paper viewed from the edge in real life. This is why 2D-MiraGe was designed for compatibility with 2D engines, similar to Graphine-MiraGe, which excels in layered editing scenarios.

Amorphous-MiraGe, on the other hand, is not strictly confined to a single plane, giving it a natural 3D effect. From a practical standpoint, this makes it the most intuitive to edit in 3D space. If only 2D edits, such as simple rotations in one plane, were applied, there would be no noticeable difference between the model versions. This distinction is illustrated in Fig. 6. However, we acknowledge that the image is too small, and we will enlarge it to improve clarity. We thank Reviewer for pointing this out.

We hope our explanation is sufficient, and we remain open to more questions.

W: (...) editing those Gaussians is quite trivial (..)

As 3D Gaussian editing still advances, so do methods for reducing artifacts. A recent study (arXiv 03.2025) [1b] enforces spherical shapes, while PhysGaussian [2b] uses an Anisotropy Regularizer—both preventing sharp artifacts during rotations. In 2D achieving high-quality and robust nonlinear modifications images is still challenging, especially when performing affine transformations on low-resolution images, where inaccuracies can lead to the appearance of holes. However, with Gaussians, affine transformations preserve key properties such as color and opacity, allowing us to avoid numerical errors and eliminate this issue. Moreover, the additional 3rd dimension can simplify animation creation in certain cases. For example, it facilitates the natural motion of a waving flag as illustrated in Fig. 3. We hope that the use of Gaussians for 2D image representation will continue to evolve, leading to more advanced and mature editing tools in the future. Our work demonstrates that such editing is not only possible but also promising.

OC&S: (...) I am concerned whether ICML is a good fit for this work...

In our paper, we introduce a novel approach to 2D image representation, specifically designed for manual editing and integration with physical engines. As the reviewers pointed out, our method lacks direct competitors - a fact that, in our view, underscores its originality.

In the reconstruction task, our method achieves superior results in a shorter time. As discussed in our response to Reviewer u88z, W2, our approach outperforms the baselines in PSNR with just 5k iterations Moreover, MiraGe enables a range of capabilities:

• Integrating physical engines with 2D images
• Editing 2D images directly within a 3D space
• Enabling complex nonlinear modifications of 2D images

Therefore, we believe ICML is a suitable venue for this work, as we present a new approach to representing 2D images.

We are committed to improving our paper and greatly appreciate the feedback regarding the visibility of Fig. 6. We will address this concern and make the necessary adjustments to the camera-ready version.

We appreciate the Reviewer's thoughtful feedback. We are pleased for the recognition of the distinct advantage of our proposed method in seamlessly integrating with physics engines, allowing for more dynamic and realistic modifications compared to traditional tools.

W1/2

In our paper, we address two tasks. The first, reconstruction, is evaluated using widely recognized benchmarks and metrics, as noted by Reviewer u88z, who found the methods and evaluation criteria appropriate.

In the second editing task, we focused on qualitative assessment, acknowledging the limitations of our evaluation as the first to apply parametric 3D Gaussians to 2D images. Nevertheless, we compare our approach to generative methods to highlight differences in editing capabilities, aiming to introduce a novel 2D representation and inspire future advancements.

W3

In most generative methods, users have constrained control over edits. However, as shown in Fig. 10, our approach allows precise modifications, enabling users to adjust specific details. Additionally, we incorporate 3D representation and human perspective (Fig. 3), demonstrating that our method inherently possesses a 3D structure. This means that, beyond standard 2D transformations like rotation, we also have control over perspective.

In PhysGen authors claim that “despite advancements in video generative models the incorporation of real-world physics principles into the video generation process remains largely unexplored and unsolved.” We show that our method can leverage physics engines, producing videos from 2D images without the need for generative model predicting the simulation (see visualization in supplementary materials). We show that using Taichi to apply materials such as sand to objects, as demonstrated in Fig.7.

Q1

Our goal is not to compete with tools like Photoshop but to highlight the advantages of representing 2D images with parameterized 3D Gaussians. While Photoshop allows for seamless image modifications, it relies on numerous hidden features such as specialized interpolation, imputation, and other corrections. When applying a simple affine transformation to an image, artifacts often emerge, necessitating pixel interpolation to correct them. In contrast, Gaussian components inherently preserve their structure under affine transformations, as an affine transformation of a Gaussian remains a Gaussian. For this reason, we believe directly comparing MiraGe to the Photoshop tool would be inappropriate.

However similar to Photoshop layering, our image representation can incorporate the idea of image layers (as illustrated in Fig. 4). This allows for inpainting when parts of the image are occluded—for example, seamlessly inpainting in the background, as demonstrated in Fig. 11.

It is worth mentioning that our model builds on features from 3DGS, enabling the application of existing Gaussian-based models. We believe that in practice, MiraGe can incorporate various style transfer methods for Gaussians. For instance, StyleGaussian [1c] allows for color adjustments during style transfer. However, such modifications fall outside the scope of this paper, as our focus is on shape editing rather than texture or color alterations. We recognize this as a promising future research direction.

[1c] K. Liu, et al. "StyleGaussian: Instant 3d style transfer with gaussian splatting." SIGGRAPH Asia 2024

Q2

Due to the character limitation, answer to this question is referenced in W1 and Q4, Reviewer u88z

Q3

We consider this an interesting question, as also noted by Reviewer u88z. We will address this in the camera-ready version.

As part of our rebuttal, we conducted an experiment on a single image to illustrate how method scale. We selected a butterfly from the DIV2K dataset[2c]. For Training we used 100K initial Gaussians;5k, 10k, 30k iterations on the V100 GPU. In table we report compressed memory (MB) and training time (seconds).

From the table below, we can observe that our method consistently achieves higher PSNR results compared to the baseline.

[2c] https://data.vision.ee.ethz.ch/cvl/DIV2K/

In practice, for high-resolution images, GS can be initially trained at a lower resolution and progressively refined to higher resolutions, following the Hierarchical GS strategy [3c], which we believe can be applied as a future work.

[3c] B. Kerbl, et al. "A hierarchical 3d gaussian representation for real-time rendering of very large datasets." ACM Transactions on Graphics, 2024

We thank the Reviewer for the feedback and constructive remarks regarding our paper that we believe will improve our paper. In particular, we are grateful for the Reviewer’s recognition that "the methods and evaluation criteria employed are appropriate and well-chosen".

W1

We acknowledge that significant edits can produce visual artifacts in the current version of our model and discuss it in in the Limitation Section. Nevertheless, our primary focus is on demonstrating the advantages of the proposed representation of 2D images using parametrized Gaussians and proving that such editing is feasible. We believe this work opens the door to numerous future research directions and improving even substantial edits.

Q4

It is a non-trivial question requiring substantial consideration.

As the field of editing 3D Gaussians continues to develop, so do the methods for mitigating artifacts appearing in 3D objects. For example, a recent study (arXiv 03.2025) [1b] introduces a loss function that enforces Gaussians to maintain spherical shapes. In PhysGaussian the authors use Anisotropy Regularizer [2b]. In both cases this helps prevent the creation of "sharp" visible artifacts during rotations.

In the context of 2D image “we introduced the mirror camera to ensure that Gaussians remain confined within a specific spatial region between the cameras, enhancing control and precision.” This approach effectively serves as a form of Gaussian regularization, as seen in Fig. 6 (first and second rows). To enhance readability, we will enlarge this figure.

It is also important to highlight that artifacts may arise naturally when working with 2D representations, such as fading effects that occur with significant changes in perspective (e.g., rotations of 90 degrees in 3rd dimension). A useful analogy is a piece of paper or a photograph: when rotated 90 degrees in the third dimension, it effectively disappears from view.

[1b] L Qiu, et al. ; “LHM: Large Animatable Human Reconstruction Model from a Single Image in Seconds”; arXiv 2025

[2b] T. Xie, et al.; “PhysGaussian: Physics-Integrated 3D Gaussians for Generative Dynamics”; CVPR 2024

W2

Below we present the supplementation of Tab. 1 and a comparison with existing methods. The metrics for the baselines were taken from [3b]; however, we re-ran the experiments for GaussianImage (GI) -70K (GI-70K) and our using a V100 GPU to ensure a more reliable comparison. Those experiments are denoted in the following table by “*” next to the method name.

We include the table with training time comparison including multiple baseline methods. The time provided for our method is the full training time (30000 and 5000 iteration steps). Additionally, using the butterfly image from the DIV2K dataset as an example, we demonstrated that our method (Our-100K) achieves higher PSNR than GI in just 30 seconds, see Fig. 3. This is also supported by our method surpassing GI in PSNR with only 5k iterations, while also having a shorter training time.

Q1

While the inference of our method is fast, the storage cost introduced by the original 3DGS representation used by our method is high. To overcome this we use the existing compression tool spz [3b]. This allows us to reduce the memory overhead by up to 95%; please see Tab. 4 in supplementary material. Additionally we include GI as our baseline. Note that GI compresses the Gaussian representation as part of their pipeline, and spz compression algorithm is not applicable to their representation:

[3b] https://github.com/nianticlabs/spz

Q2

Due to the character limitation, answer to this question is referenced in Q3, Reviewer VWK2

Q3

Indeed, in complex scenes with intricate backgrounds, the task becomes more challenging, as we mention in the Limitations Section. However, as shown in Fig. 11, our method allows for seamless object manipulation, such as repositioning a flower using a physic engine such as Taichi Elements, while effectively applying inpainting to ensure a gap-free result.

We sincerely thank the Reviewer for their valuable feedback and are pleased with the appreciation of our work. In particular, we are especially grateful for the recognition of the breadth and depth of our experiments "covering multiple datasets and providing both quantitative and qualitative comparisons."

W1. A small problem is that the author could add the PSNR/SSIM score on Fig.5.

Thank you for this suggestion. We will add the mentioned metrics to the Fig. 5 in the camera-ready version:

C&S1. Does the MiraGe structure require training on a single image each time? Would this be considered low-efficiency?

The pipeline does require training for each image individually. However, it is important to note that even with just 30 seconds of training, our method already produces better results compared to existing approaches (Fig. 9; Our-100k, using the butterfly image from DIV2K as an example). At the same time, longer training is beneficial for more-optimal PSNR performance. For more comparison, please refer to our answer to Reviewer u88z, W2 where we show that our method is surpassing baselines in PSNR with only 5k iterations, while also having a shorter training time.

C&S2. What plans do the authors have for improving this aspect in the future?

A possible direction to improve our approach with regards to training a different model for each individual image, could be the use of a generative model to prepare the collection of Gaussians based on the input image. While literature explores such an approach to the best of our knowledge current models work on foreground objects, and not full scenes like [1a]. In our case the images used are real life photographs, where both foreground and background are required to be reconstructed. Additionally, we would like to highlight that our model rapidly obtains high quality reconstruction.

[1a] TANG, Jiaxiang, et al. DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creation. In: The Twelfth International Conference on Learning Representations.

Q1. If more physical factors, such as real-world lighting, BRDF, and materials, are considered, would this enhance the performance of the work? If it possible to incorporate these factors to improve MiraGe's performance? In other words, does MiraGe have the potential to expand in the direction of inverse rendering (i.e. NeRFactor, Ref-NeRF)?

Thank you for posting this interesting question. In literature there exist works that incorporate real-world lighting using 3D Gaussian Splatting [2a, 3a]. In contest to BRDF, 3D Gaussian Ray Tracing [4a] uses Ray Tracing to enable secondary lightning effects. Since all the mentioned methods (including ours) are based on 3D Gaussian Splatting, we believe that integrating similar ideas into MiraGe is possible. We consider this a promising direction for future work, which we will post in the main manuscript.

[2a] J. Gao, et al. “Relightable 3D Gaussians: Realistic Point Cloud Relighting with BRDF Decomposition and Ray Tracing”; ECCV2024

[3a] Z. Bi, et al. GS^3: “Efficient Relighting with Triple Gaussian Splatting”; SIGGRAPH Asia 2024

[4a] N. Moenne-Loccoz, et al.; “3D Gaussian Ray Tracing: Fast Tracing of Particle Scenes”; SIGGRAPH Asia 2024