MeshLRM: Large Reconstruction Model for High-Quality Meshes

Comparison with newer baseline (W10): We have added a new table to quantitatively compare our method against recent baselines. Given that input settings differ among the baselines, we follow the setup in the concurrent work MeshFormer [4] to evaluate all methods under a unified single-view-to-3D setting. Specifically, for InstantMesh, MeshFormer, and our method, we first used Zero123++ [5] to convert the input single-view image into multi-view images before performing 3D reconstruction. The other baselines follow their original configurations, taking a single-view image directly as input.

The results, where all methods generate a mesh as the final output, are presented below.

Our method demonstrates a significant advantage over TripoSR, LGM, and InstantMesh across all three metrics on both datasets. Additionally, it is comparable with MeshFormer, with slightly lower PSNR and better SSIM and LPIPS. Note that MeshFormer is specifically designed for the single-image-to-3D task, it requires multi-view consistent normals as input during its reconstruction step. This makes it unsuitable for handling sparse-view reconstruction scenarios using captured images, such as setups with OpenIllumination, where only RGB images with camera poses are available. In contrast, our method can well handle this as shown in Table 6.

We will also expand the discussion on recent works in the related work section and include the new results in the revised version.

What makes density fields preferable in this context? (Q1) Based on the study on per-scene optimization papers, SDF-based methods, suggested by the reviewer, tend to produce over-smoothed geometry and struggle to capture thin structures, as noted by Nerf2Mesh [7], and they also achieve worse rendering quality compared to density-based methods. In contrast, the strategy of pretraining with density fields followed by fine-tuning with rasterization has been demonstrated to be effective in previous works, such as Magic3D [6], NeRF2Mesh [7], and NeuManifold [8]. Therefore, we have chosen to adopt this approach. We successfully make our mesh rendering quality match the volume rendering quality, which is rare in per-scene optimization methods.

The reason for choosing DiffMC (Q2): Our pipeline is compatible with DMTet and Flexicubes, not limited to DiffMC. However, we chose DiffMC due to its superior processing speed and minimal GPU VRAM consumption (as detailed in the comparison table on its GitHub repository: https://github.com/SarahWeiii/diso?tab=readme-ov-file#speed-comparison). This efficiency is particularly beneficial for large-scale training, where both speed and GPU optimization are crucial.

Compared to the concurrent work InstantMesh, there are numerous differences between our pipeline and theirs. Factors such as model architecture, model size, training precision, and other design choices can significantly impact the training difficulty. However, we have carefully addressed the challenges in training our model, resulting in a method that achieves better quality than InstantMesh [3].

[1] Tochilkin, Dmitry, et al. "Triposr: Fast 3d object reconstruction from a single image." arXiv (2024).

[2] Tang, Jiaxiang, et al. "Lgm: Large multi-view gaussian model for high-resolution 3d content creation." ECCV, 2025.

[3] Xu, Jiale, et al. "Instantmesh: Efficient 3d mesh generation from a single image with sparse-view large reconstruction models." arXiv (2024).

[4] Liu, Minghua, et al. "Meshformer: High-quality mesh generation with 3d-guided reconstruction model." NeurIPS (2024).

[5] Shi, Ruoxi, et al. "Zero123++: a single image to consistent multi-view diffusion base model." arXiv (2023).

[6] Lin, Chen-Hsuan, et al. "Magic3d: High-resolution text-to-3d content creation." CVPR. 2023.

[7] Tang, Jiaxiang, et al. "Delicate textured mesh recovery from nerf via adaptive surface refinement." ICCV. 2023.

[8] Wei, Xinyue, et al. "Neumanifold: Neural watertight manifold reconstruction with efficient and high-quality rendering support." arXiv (2023).

[9] Munkberg, Jacob, et al. "Extracting triangular 3d models, materials, and lighting from images." CVPR. 2022.

Abbreviations in the answers: W: Weaknesses; Q: Questions

Clarification about post-optimization (W1, W3): Post-optimization in Line 56 refers to techniques applied after training a neural radiance field, requiring a separate mesh-related optimization stage, such as fine-tuning (nerf2mesh [1], NeuManifold [2]) or distillation (Nerfmeshing [3]), to enhance mesh reconstruction quality in the per-scene optimization case. It does not refer to directly applying marching cubes to the density field.

Why converting NeRF-based method leads to performance drop (W2): Since volume rendering does not require a strict surface definition, applying marching cubes to extract a mesh surface can easily lead to inaccuracies in mesh quality, such as geometric noise or artifacts (e.g., ringing effects). This is why previous works [1][2][3] try to address this in the per-scene optimization task.

Why NeRF-based pretraining is necessary (W5): We found that single-stage training (i.e., without NeRF initialization) fails to produce reasonable results. Directly training using a mesh representation often gets stuck in local minima and fail to converge, whereas neural radiance fields provide a better representation for handling topological changes in a smooth, continuous manner. This issue has also been widely noted and discussed in several per-scene optimization studies, such as Magic3D [1], NeuManifold [2], and Nerf2Mesh [3].

Statement of end-to-end optimization (W6): Although our training process is divided into two stages, the network architecture remains consistent. And our second-stage mesh training optimizes all components jointly, directly regressing the final mesh output from the input images, which is completely end-to-end.

Why DINO is not necessary (W7): In addition to comparing with In3D, we also evaluated the benefits of DINO features and trained a MeshLRM model using the DINO encoder for the same number of steps as our proposed models, which leads to negligible quality differences. More details can be seen in paper line 405.

Quantitative comparison between NeRF-based LRM and Mesh-based LRM (W8): We provide a quantitative comparison in Table 5 between MeshLRM (NeRF) and MeshLRM. Our MeshLRM achieves comparable results to MeshLRM (NeRF), with even better performance on SSIM and LPIPS metrics. This is particularly challenging because, in per-scene optimization studies, mesh-based methods typically exhibit lower quality compared to volumetric-based methods, as shown in nvdiffrec [9] and nerf2mesh [7].

Quantitative comparison in the progressive training regarding resolution (W9): We provide a quantitative comparison in Table 1, contrasting training from scratch on high-resolution data with our low-to-high-resolution training strategy.

After reading through the responses and review suggestions from other reviewers, I noticed that all reviewers raised concerns about the novelty of this work. Reviewer ftqh specifically pointed out the lack of public availability.

While the authors argue that the simplification of the network architecture and the regularization loss are carefully designed, I believe that without DiffMC and using Flexicube as InstantMesh, we wouldn’t need the regularization loss at all.

Furthermore, although Reviewer 6QBv mentions that "While combining LRM with diffMC is an effective and practical engineering approach, the concept appears somewhat ad hoc. However, this does not constitute a significant reason for rejection, as the paper still meets the typical standards of previously accepted work.'', I do not fully agree with this point. I do not think that LRMs or 3D generation should be held to a lower standard for acceptance.

Overall, considering all of these factors, we have decided to revise my rating to a weak reject.

We are glad to have addressed the earlier concerns raised by the reviewer during the discussion. However, we are very surprised that new concerns, which were not previously mentioned or emphasized, have now been introduced and considered significant enough to lower the rating. We now provide quick responses to address these points.

Public Availability: The reviewer has already discovered and successfully tested a publicly available unofficial demo of our method, which shows similar quality. This clearly demonstrates the reproducibility and availability of our approach. We promise to release a public demo implemented by the authors to ensure availability.

Novelty and Contribution: While we acknowledge varying perspectives on assessing a paper’s contribution, we believe the most critical measure—and the fundamental goal of research—is advancing the state of the art in the field. Our work achieves this by outperforming all baseline methods, including very recent approaches such as LGM (ECCV 2024) and MeshFormer (NeurIPS 2024), both of which were recognized as state of the arts and published within months. This highlights the significance of our contribution. We believe that any work that genuinely advances the state of the art with carefully justified experiments deserves to be seen by the community. Furthermore, we strongly disagree with the characterization of our work as "ad hoc" or "trivial engineering." Every design choice in our framework is carefully evaluated and supported by experimental evidence. If any aspect remains unclear, we are happy to provide additional clarifications.

Regarding InstantMesh: InstantMesh is a concurrent work with high-level similarities to ours. In principle, such concurrent works should not influence the evaluation of our paper. Nevertheless, we have already provided extensive discussions and comparisons with InstantMesh, demonstrating that our method achieves superior quality. Specifically, we emphasize that our opacity loss is a novel design in our pipeline, as shown in newly added Figure 5, with experimental evidence in Figure 3 confirming its necessity and effectiveness. Designing a new model that integrates elements from both our work and InstantMesh to achieve high quality while removing the opacity loss goes beyond the scope of this submission.

We believe unbiased reviewing should focus on the demonstrated quality, experiments, and contributions of the submission itself. If there are specific points that remain critical to alter the rating, we kindly ask for clarification so we can address them in detail.

Abbreviations in the answers: W: Weaknesses; Q: Questions

Statement about “directly output meshes” (W1): Our method is indeed different from MeshGPT, which directly predicts triangles. However, our approach incorporates marching cubes within its pipeline and employs an explicit mesh representation during training. Therefore, we consider our method to be directly generating meshes, unlike other LRM methods that rely on implicit representations during training and require additional processing to extract a mesh. We will add further clarification to clearly distinguish our approach from MeshGPT-like methods to make the claim clearer.

Novelty of the paper (W2): The novelty of the paper is not limited to combining LRM and differentiable iso-extraction, but also proposing a simplified version of LRM architecture, which is mentioned to be “important and directly applicable to other LRM-based approaches” by Reviewer apsR. We show that a simple transformer is effective for 3D reconstruction, offering an easier path to building and scaling LRMs. Moreover, combining LRM and DiffMC in an end-to-end training framework encounters new challenges (like unstable training) and requires novel solutions. Our carefully designed training strategies and losses are crucial for ensuring stable training and high performance. We hope our findings can benefit the community in building scalable 3D reconstruction models. We sincerely appreciate the reviewer’s recognition that our paper “meets the typical standards of previously accepted work”!

Ablation study on fine-tuning (Q1): We experimented with fine-tuning only part of the architecture and observed that all the losses converged to higher values compared to fine-tuning the entire network. In particular, the ringing artifacts on the geometry were still difficult to eliminate. Therefore, we choose to fine-tune the entire pipeline.

Abbreviations in the answers: W: Weaknesses; Q: Questions

Comparison with newer baseline and on real-world dataset (W1, W2): We have added a new table to quantitatively compare our method against recent baselines both on GSO and OmniObject3D datasets. Given that input settings differ among the baselines, we follow the setup in the concurrent work MeshFormer [4] to evaluate all methods under a unified single-view-to-3D setting. Specifically, for InstantMesh, MeshFormer, and our method, we first used Zero123++ [5] to convert the input single-view image into multi-view images before performing 3D reconstruction. The other baselines follow their original configurations, taking a single-view image directly as input.

The results, where all methods generate a mesh as the final output, are presented below.

Our method demonstrates a significant advantage over TripoSR, LGM, and InstantMesh across all three metrics on both datasets. Additionally, it is comparable with MeshFormer, with slightly lower PSNR and better SSIM and LPIPS. Note that MeshFormer is specifically designed for the single-image-to-3D task, it requires multi-view consistent normals as input during its reconstruction step. This makes it unsuitable for handling sparse-view reconstruction scenarios using captured images, such as setups with OpenIllumination, where only RGB images with camera poses are available. In contrast, our method can well handle this as shown in Table 6.

We will also expand the discussion on recent works in the related work section and include the new results in the revised version.

Technical novelty (W3): The novelty of our paper extends beyond simply combining LRM with differentiable iso-extraction. We also introduce a simplified version of the LRM architecture, which, as noted by Reviewer apsR, is “important and directly applicable to other LRM-based approaches.” We demonstrate that a straightforward transformer can be highly effective for 3D reconstruction, providing a more accessible path to building and scaling LRMs.

Training 3D LRMs presents significantly more challenges compared to per-scene optimization (e.g., Magic3D). Techniques like differentiable rasterization, when applied in this context, often face new issues such as unstable training, necessitating novel solutions. Our carefully designed training strategies and loss functions are critical for achieving stable training and high performance. We hope our findings will benefit the community in developing scalable 3D reconstruction models.

Furthermore, compared to concurrent work like InstantMesh, our method achieves superior quality as shown in the comparison table provided above.

Training stability (W3): The limitation of directly training on mesh representations is not only due to training stability, but also because mesh representations are more prone to getting stuck in local minima during optimization. In contrast, neural radiance fields are a better choice as they can handle topological changes in a smooth and continuous manner, as noted in studies like Magic3D [6] and NeuManifold [7]. Our experiments also confirm this: single-stage training (i.e., without NeRF initialization) fails to produce reasonable results.

[1] Tochilkin, Dmitry, et al. "Triposr: Fast 3d object reconstruction from a single image." arXiv (2024).

[2] Tang, Jiaxiang, et al. "Lgm: Large multi-view gaussian model for high-resolution 3d content creation." ECCV, 2025.

[3] Xu, Jiale, et al. "Instantmesh: Efficient 3d mesh generation from a single image with sparse-view large reconstruction models." arXiv (2024).

[4] Liu, Minghua, et al. "Meshformer: High-quality mesh generation with 3d-guided reconstruction model." NeurIPS (2024).

[5] Shi, Ruoxi, et al. "Zero123++: a single image to consistent multi-view diffusion base model." arXiv (2023).

[6] Lin, Chen-Hsuan, et al. "Magic3d: High-resolution text-to-3d content creation." CVPR. 2023.

[7] Wei, Xinyue, et al. "Neumanifold: Neural watertight manifold reconstruction with efficient and high-quality rendering support." arXiv (2023).

Abbreviations in the answers: W: Weaknesses; Q: Questions

Ray opacity loss v.s. other regularization (W1, Q2): The main difference between our ray opacity loss and previous approaches lies in the selection of supervision points. Standard density regularization uniformly supervises all points in space, including those on the mesh surface as well as in empty regions, which can lead to gradients that are less effective at guiding the model toward a clean surface. On the other hand, standard depth loss only supervises points on the mesh surface; once floaters appear during training, its gradients can move these floaters but cannot easily eliminate them. In contrast to these standard losses, our opacity loss supervises the entire space between the surface and the camera, effectively preventing floaters and stabilizing the training process. We will add a figure in the revised paper to further illustrate the difference.

Evidence of choosing DiffMC (W2, Q1): Our pipeline is compatible with DMTet and Flexicubes, not limited to DiffMC. However, we chose DiffMC due to its superior processing speed and minimal GPU VRAM consumption (as detailed in the comparison table on its GitHub repository: https://github.com/SarahWeiii/diso?tab=readme-ov-file#speed-comparison). This efficiency is particularly beneficial for large-scale training, where both speed and GPU optimization are crucial.

Performance compared to recently published baselines (W3, W4, Q4): We have added a new table to quantitatively compare our method against recent baselines. Given that input settings differ among the baselines, we follow the setup in the concurrent work MeshFormer [4] to evaluate all methods under a unified single-view-to-3D setting. Specifically, for InstantMesh, MeshFormer, and our method, we first use Zero123++ [5] to convert the input single-view image into multi-view images before performing 3D reconstruction. The other baselines follow their original configurations, taking a single-view image directly as input.

The results, where all methods generate a mesh as the final output, are presented below.

Our method demonstrates a significant advantage over TripoSR, LGM, and InstantMesh across all three metrics on both datasets. Additionally, it is comparable with MeshFormer, with slightly lower PSNR and better SSIM and LPIPS. Note that MeshFormer is specifically designed for the single-image-to-3D task, it requires multi-view consistent normals as input during its reconstruction step. This makes it unsuitable for handling sparse-view reconstruction scenarios using captured images, such as setups with OpenIllumination, where only RGB images with camera poses are available. In contrast, our method can well handle this as shown in Table 6.

We will also expand the discussion on recent works in the related work section and include the new results in the revised version.

Density threshold (Q3): Both DiffMC and standard Marching Cubes use the same density threshold, which is selected based on the overall quality of the Marching Cube mesh. During stage two fine-tuning, DiffMC maintains this threshold. In general, we observe that, without DiffMC fine-tuning, the Marching Cubes artifacts shown in the paper are unavoidable with various thresholds. While carefully tuning the threshold for a single scene can reduce the artifacts to some extent, the optimal threshold differs notably across scenes. Our DiffMC fine-tuning effectively resolves these issues, allowing for high-quality mesh reconstruction with the consistent pre-selected threshold.

[1] Tochilkin, Dmitry, et al. "Triposr: Fast 3d object reconstruction from a single image." arXiv (2024).

[2] Tang, Jiaxiang, et al. "Lgm: Large multi-view gaussian model for high-resolution 3d content creation." ECCV, 2025.

[3] Xu, Jiale, et al. "Instantmesh: Efficient 3d mesh generation from a single image with sparse-view large reconstruction models." arXiv (2024).

[4] Liu, Minghua, et al. "Meshformer: High-quality mesh generation with 3d-guided reconstruction model." NeurIPS (2024).

[5] Shi, Ruoxi, et al. "Zero123++: a single image to consistent multi-view diffusion base model." arXiv (2023).