Restore3D: Breathing Life into Broken Objects with Shape and Texture Restoration

We sincerely appreciate your recognition of our methodology and your positive feedback on the details, experiments, and analysis. Thank you very much for your valuable and encouraging comments!

We will address all the concerns point by point.

W1 & Q1.

1. The framework primarily consists of multiview inpainting and reconstruction, which makes it relatively straightforward from this perspective. We will clarify it in the revision.

2. Previous super-resolution is not unnecessary. The high-resolution images refined by super-resolution are used in the geometry and texture refinement stage to improve the details of the coarse meshes inferred by LRM. As shown in Figure 8(a) and Table 2(b) of our paper, the high-resolution images play an important role in coarse mesh refinement.

3. Why not train a model to inpaint the model and do super-resolution at the same time?

   • Inpainting and super-resolution have inherently different objectives and feature demands. Inpainting focuses on semantic hole-filling and structural reconstruction, requiring strong contextual modeling and high-level understanding of scene semantics. In contrast, super-resolution emphasizes pixel-level detail enhancement and high-frequency texture recovery. These intrinsic differences make it challenging to design a unified architecture that performs both tasks effectively.
   • Training a single model to handle both inpainting and super-resolution significantly increases computational demands. In multi-view settings, these costs scale even more severely compared to single-view configurations. Furthermore, as resolution increases, the computational cost grows exponentially. Consequently, the resolution of the output will be constrained to a relatively low level and still require the addition of a secondary model for further high-resolution enhancement.
   • Decoupling inpainting and super-resolution into separate, modular components enables greater flexibility and maintainability. Each module can be independently developed, fine-tuned, or replaced with more advanced techniques as they become available.

4. Comparison with direct3d and trellis. We input the front-view image inpainted by our model into direct3d and trellis. Since Direct3D does not release its texture generation code, we report only geometry results for that method. The results show that our model outperforms both methods. These baselines are designed for single-view reconstruction, which naturally limits their performance compared to our approach that leverages multi-view inputs. We will add this comparison in the revision. | Model | CD |F-score| PSNR | LPIPS | |----------|------|-------|------|-------| | Direct3D | 0.006 | 0.297 | - | - | | Trellis | 0.005 | 0.335 | 21.78 | 0.12 | | Ours | 0.005 | 0.389 | 23.35 | 0.09 |

W2. Camera. Yes, the pipeline requires a known camera during evaluation. The source of "Camera Embeddings" in Figure 3 is the known 4-view camera parameters. We will clarify it in the revision.

W3. The cross-section is too smooth and regular, and may not be able to fully reflect the missing appearance of real objects.

We conduct experiments on the following datasets. As shown in Figure 11 of our paper and Figure 1 of our supplementary material, both of these datasets include non-regular break patterns, further validating the robustness of our approach.

1. Synthetically fractured objects. As shown in Table 5 and Figure 11 of our paper, we test our model on Breaking Bad Dataset[1]. This dataset is synthesized by a physically based method that simulates the natural destruction process of geometric objects.

2. Real-world objects. In Figure 1 and Table 1 of our supplementary material, we include experimental results regarding Fantastic Breaks[2]. This dataset contains 150 paired real-world broken objects.

These experiments demonstrate the generalization ability of our model to both real-world scenarios and unseen physically simulated cases, validating its robustness and practical applicability.

Thank you for highlighting this issue. We will reorganize these contents to enhance their visibility and ensure better clarity.

[1] Breaking bad: a dataset for geometric fracture and reassembly. NIPS 2022.

[2] A dataset of paired 3d scans of real-world broken objects and their complete counterparts. CVPR 2023.

W4. Typos. "Complete3D" is "Restore3D"; we will correct this in the future version of the paper.

We sincerely appreciate your recognition of the novelty of our designed method and your positive feedback on the details, experiments, and analysis. Thank you very much for your valuable and encouraging comments!

We will address all the concerns point by point.

W1. Inconsistency of motivation and experiments.

1. Our primary contribution is that we are among the first to explore the restoration or completion of the shape and texture in broken 3D objects. Based on this, we propose a well-structured and effective pipeline tailored for this task. We conduct comprehensive experiments to assess the performance and generalization capability of our model across diverse scenarios.

   • As shown in Table 1 and Figures 6 of our paper, we evaluate our model on datasets generated using the same synthetic method employed during training.
   • As shown in Table 5 and Figure 11 of our paper, we further test our model on the Breaking Bad Dataset[1] synthesized by a physically based method, different from the synthesis method used in our training.
   • As shown in Table1 and Figure 1 of the supplementary material, we also evaluate our model on real-world broken objects.

   These experiments demonstrate the generalization ability of our model to both real-world scenarios and unseen physically simulated cases, validating its robustness and practical applicability.

2. Occlusion handling is not a primary focus of our work. It is a byproduct of our model's design, as a minor contribution, evaluated in Table 4 and Figure 10.

Thank you for highlighting this issue. We will reorganize these contents to enhance their visibility and ensure better clarity.

W2. Experiments on real or synthetically fractured objects

1. Synthetically fractured objects. As shown in Table 5 and Figure 11 of our paper, we test our model on Breaking Bad Dataset[1]. This dataset is synthesized by a physically based method that simulates the natural destruction process of geometric objects.

2. Real-world objects. In Figure 1 and Table 1 of our supplementary material, we include experimental results regarding Fantastic Breaks[2]. This dataset contains 150 paired real-world broken objects.

Both of these results show the generalization ability of our model. Thank you for highlighting this issue. We will reorganize these contents to enhance their visibility and ensure better clarity.

[1] Breaking bad: a dataset for geometric fracture and reassembly. NIPS 2022.

[2] A dataset of paired 3d scans of real-world broken objects and their complete counterparts. CVPR 2023.

W3. Comparison with coarse-to-fine methods.

1. Coarse-to-fine is not our main contribution. Our primary contribution is that we are among the first to explore the restoration or completion of the shape and texture in broken 3D objects. Based on this, we propose a well-structured and effective pipeline tailored for this task. Central to our method is the Mask Self-Perceiver module, which generates masks that guide a consistent super-resolution process. This results in high-quality image outputs, enabling robust multi-view consistent inpainting.
2. Coarse-to-fine plays a crucial role, as shown in Figures 4 & 8 of our paper. By progressively refining the reconstruction, it significantly enhances performance and leads to more faithful and realistic restoration results. Thus, it remains a necessary and impactful design choice within our overall pipeline.

• 3DEnhancer. 3DEnhancer is a multi-view image enhancement method. We compare it with our image integration and enhancement pipeline, and our results demonstrate superior performance. It is worth noting that 3DEnhancer is limited to a resolution of 512 pixels, whereas our method supports resolutions of 1K and can be scaled up to 2K and even 4K. Moreover, 3D Enhancer can not preserve the texture of the original broken part if it does not use our predicted mask. | model | PSNR |LPIPS|SSIM| |-|-|-|-| | 3D Enhancer| 26.37 | 0.07 | 0.96 | | Ours | 26.97| 0.05| 0.97|

• Unique3D. We replace the single-image to multi-view model in Unique3D with our inpainting model. Experimental results indicate that our method achieves better reconstruction performance. | Model | CD |F-score| PSNR | LPIPS | |----------|------|-------|------|-------| | Unique3D | 0.005 | 0.306 | 22.00 | 0.14 | | Ours | 0.005 | 0.389 | 23.35 | 0.09 |

• Sharp-It. The author does not release their code, so we are unable to conduct a direct comparison. We will cite their work and discuss it in the Related Work section.

Thank you for highlighting this issue. We will include these experiments and discuss them in the related work.

Q1: Yes. The input of the other methods is the inpainted and enhanced images generated by our model.

Q2: In our opinion, for per-object reconstruction, the training time refers to the optimization of both geometry and texture, while the inference time corresponds to the rendering of the final mesh.

Dear Reviewer,

I would like to invite you to the discussions with the authors. At least, please carefully read the others' reviews and authors' responses, and mention if the rebuttals addressed the concerns or not.

To facilitate discussions, the Author-Reviewer discussion phase has been extended by 48h till Aug 8, 11:59 pm AoE; but to have enough time to exchange opinions, please respond as quickly as possible.

Thanks,

Your AC

We sincerely appreciate your recognition of our methodology and the proposed task. Thank you very much for your valuable and encouraging comments!

Our core contribution is that we are among the first to explore the restoration or completion of the shape and texture in broken 3D objects. Based on this, we develop a data synthesis pipeline to construct training pairs and propose a well-structured and effective pipeline tailored for this task. Extensive experiments show the performance and generalization capability of our model across unseen physically simulated cases and real-world cases.

We will address all the concerns point by point.

W1 & Q1. Comparison with multiview inpainting methods

1. As shown in Table 1 and Figure 6 of our paper, we include comparative results with the multi-view inpainting method Nerfiller [1], not limited to single-view settings.
2. Additionally, we present new comparisons with Instant3dit [2], a multi-view inpainting method. Unlike our approach, Instant3dit requires a manually defined mask to specify the inpainting region. When we use a simple inverted mask of the broken object as input, Instant3DIt performs noticeably worse than our method. When leveraging the inpainting mask predicted by our model, Instant3dit sees significant performance improvements. Even so, it continues to underperform relative to our approach. | model | PSNR |LPIPS|SSIM| |-|-|-|-| | Instant3edit | 19.40 | 0.10 | 0.94 | | Instant3edit (w/ our predicted mask) | 22.37 | 0.07 | 0.95 | | Ours | 25.50| 0.06| 0.95|

   [1] Nerfiller: Completing scenes via generative 3d inpainting. CVPR, 2024.

   [2] Instant3dit: Multiview Inpainting for Fast Editing of 3D Objects. CVPR 2025.

   We will clarify it and include these experiments in the revision.

W2. Fonts and diagrams.
We apologize for the poor quality of the figures and fonts in the charts.

• We will update the fonts.
• We will adjust the text that currently covers the images.
• Regarding the figures, could you please indicate which parts are unclear and share any suggestions you may have for improvement? We understand that the multiview inpainting pipeline may seem complex, and we plan to reorganize it to enhance clarity. For example, we will first illustrate a block in Unet that has three different types of cross-attention connected by our Mask Rectifier, and then we will provide details of each cross-attention layer.

W3 & Q2 example1. Single-view inpainting + image-to-multiview.

"Single-view inpainting + image-to-multiview generation" is inherently less accurate than multiview inpainting approaches, as the synthesized views produced by image-to-multiview techniques often misalign with the ground-truth multiview images.

We first apply SD-inpainting and pix2gestalt [2] to the front-view image, followed by an SOTA image-to-multiview method, MV-Adapter[1]. However, as illustrated in Figure 2 of our paper, single-view inpainting methods often encounter specific limitations, such as Depth Understanding and Inpainting Position Perception. Consequently, the multiview images generated by MV-Adapter tend to exhibit degraded quality.

We will clarify it and include these experiments in the revision.

[1] MV-Adapter: Multi-view Consistent Image Generation Made Easy. ICCV 2025

[2] pix2gestalt: Amodal Segmentation by Synthesizing Wholes. CVPR 2024

Q2 example 2.

1. LRM is unable to accurately capture the shape of broken objects.
2. The front-view mask does not indicate the regions that require inpainting. As a result, the outputs are essentially the same as directly inputting the multiview images into the inpainting model.
3. Please refer to the example in Figure 6 of our paper (Row 3 Input). Ideally, the goal is to inpaint the gray region. However, the approach you mentioned fails to infer the correct inpainting mask corresponding to this gray area.

If we have misunderstood your idea, please feel free to correct us. We will proceed with the experiments accordingly.

Q3. Experiments on real and synthetically fractured objects.

1. Synthetically fractured objects. As shown in Table 5 and Figure 11 of our paper, we test our model on Breaking Bad Dataset[1]. This dataset is synthesized by a physically based method that simulates the natural destruction process of geometric objects.

2. Real-world objects. In Figure 1 and Table 1 of our supplementary material, we include experimental results regarding Fantastic Breaks[2]. This dataset contains 150 paired real-world broken objects.

   These experiments demonstrate the generalization ability of our model to both real-world scenarios and unseen physically simulated cases, validating its robustness and practical applicability. We will incorporate these experiments into the main paper to enhance their visibility and ensure better clarity.

   [1] Breaking bad: a dataset for geometric fracture and reassembly. NIPS 2022.

   [2] A dataset of paired 3d scans of real-world broken objects and their complete counterparts. CVPR 2023.

3. Failure cases. The main failure cases are concentrated in regions that originally contained rich details but have been severely damaged, e.g., the face or head of dolls. As a result, the inpainting process struggles to fully reconstruct these areas, often leading to suboptimal or undesirable outcomes. We will add this discussion to the limitations.

Limitation 1. For the real-world dataset Fantastic Breaks, the objects are already pre-scanned, so geometric reconstruction is not required in this case. We can easily get the 4-view images from the scanned objects. In practice, we can also take a 4-view photo using a phone, as shown in Figure 10 of our paper. It is also sufficient to achieve good reconstruction results.

Q4 & Limitations 2. High-resolution.

1. Our image integration and enhancement pipeline achieves a resolution of 1K in all experiments, as demonstrated in Figures 4 and 5 of our paper, rather than being limited to 512.
2. In most multiview settings, e.g., 3D enhancer, Unique3D, Hunyuan3D, the resolutions of the multiview model are limited to 512 or lower. After that, they will use single-view super-resolution methods (e.g., Stable-Diffusion Upscale 4x, Controlnet Tile, or RealESRGAN) to enhance resolution.
3. Our pipeline adopts Real-ESRGAN and Controlnet tile as the backbone, which is capable of scaling outputs up to 2K or 4K resolution. However, in our case, we observe no substantial improvement in visual detail at 2K compared to 1K. Furthermore, optimizing textures at 2K introduces significantly higher GPU memory consumption and training time, which we find impractical given the marginal visual gains relative to the 1K resolution.

Limitation 3. Cultural relics restoration.

Our training and testing dataset includes objects with shapes and textures similar to those of cultural relics, such as vases and statues. This research may contribute to the restoration of cultural relics.

Thank you for pointing out the limitations. We will include them in our list.

We sincerely thank you for your time and thoughtful feedback. We have carefully considered and addressed the concerns raised in our rebuttal. As the discussion period concludes in three days, please feel free to let us know if any further clarification is needed. We sincerely appreciate your engagement.

Dear Reviewer,

I would like to invite you to the discussions with the authors. At least, please carefully read the others' reviews and authors' responses, and mention if the rebuttals addressed the concerns or not.

To facilitate discussions, the Author-Reviewer discussion phase has been extended by 48h till Aug 8, 11:59 pm AoE; but to have enough time to exchange opinions, please respond as quickly as possible.

Thanks,

Your AC

We sincerely appreciate your recognition of the novelty of our proposed problem and your positive feedback on the details, experiments, and analysis. Thank you very much for your valuable and encouraging comments!

We will address all the concerns point by point.

W1. Real-world examples.

As shown in Figure 1 and Table 1 of our supplementary material, we present experimental results on the Fantastic Breaks dataset [1], which contains 150 pairs of real-world broken objects. For instance, although the fracture surfaces in this dataset are often irregular and unsmooth, our model is still able to perform promising inpainting. These results demonstrate the generalization ability of our model to real-world scenarios, despite being trained solely on synthetic data.

Thank you for highlighting the experiments involving real-world broken objects. We will incorporate these experiments into the main paper to enhance their visibility and ensure better clarity.

[1] A dataset of paired 3d scans of real-world broken objects and their complete counterparts. CVPR 2023.

W2. Physical-based material synthesis.

1. Although our model does not currently predict material attributes, our primary contribution lies in introducing a new task, simultaneously restoring the shape and texture of broken objects, and designing an effective pipeline. Hence, the objective of this task is to generate a complete, textured 3D mesh from partial observations, which is not an easy task. We also conduct extensive experiments to validate the effectiveness of our proposed pipeline.

2. At present, material attribute inpainting is significantly more difficult than simply predicting color. Designing a model capable of simultaneously inpainting material properties is inherently complex. Such a model must capture intricate interrelationships between different material attributes while preserving the structural integrity of each individual material. This increases the overall model complexity, requires more computational resources, and complicates the training process.

3. Recent studies [1,2] have shown the potential of visual foundation models in estimating material properties. We believe these models can be effectively leveraged in a post-processing stage to enhance the realism of generated textured meshes.

4. In future work, we plan to design a physics-based material inpainting model, building upon both these foundation models and the insights gained from our Restore3D pipeline.

Thank you for highlighting this issue. We will include this discussion in the limitations and future work section.

[1] Material Anything: Generating Materials for Any 3D Object via Diffusion. CVPR 2025

[2] Neural LightRig: Unlocking Accurate Object Normal and Material Estimation with Multi-Light Diffusion. CVPR 2025

W3. Efficiency.

During inference, our model does not require substantial GPU resources. A single NVIDIA RTX 3090 GPU (24GB) is sufficient for running the entire pipeline efficiently.

We report the per-object inference time for each component of our method as follows:

1. For inpainting, our base model (MVDream) is fine-tuned from Stable Diffusion (SD), and shares the same network architecture as SD. Compared to the standard SD inpainting time of 2 seconds, the additional inference time in our method is primarily due to the introduction of our proposed Mask Self-Perceiver and Cross-Attention mechanism.
2. For reconstruction, our approach requires approximately 20 seconds per object, which is higher than the LRM baseline (6s). However, this additional time is justified by the significant performance improvements achieved.

In summary, our model is computationally efficient, runs on modest GPU memory, and delivers high-quality results.

Thank you for mentioning the efficiency. We will add this discussion to the revision.

Dear Reviewer,

I would like to invite you to the discussions with the authors. At least, please carefully read the others' reviews and authors' responses, and mention if the rebuttals addressed the concerns or not.

To facilitate discussions, the Author-Reviewer discussion phase has been extended by 48h till Aug 8, 11:59 pm AoE; but to have enough time to exchange opinions, please respond as quickly as possible.

Thanks,

Your AC

Dear reviewers, Thanks so much for reviewing the paper. The discussion period ends soon. To ensure enough time to discuss this with the authors, please actively engage in the discussions with them if you have not done so.