PBR-SR: Mesh PBR Texture Super Resolution from 2D Image Priors

We sincerely thank the reviewer for the constructive feedback. We are glad that our work was recognized as addressing a 'practical and less explored problem' with 'clear motivation' and that our combination of diffusion priors, differentiable rendering, and pixel-wise supervision was found to 'significantly improve performance'. Below, we provide detailed responses to the concerns raised.

1. Optimization-based inference pipeline is relatively slow

The optimization-based inference pipeline in PBR-SR can be slower compared to fully feed-forward models. This design choice is motivated by our aim to achieve the highest possible fidelity under novel lighting conditions and get rid of extra PBR data reliance, which current learning-based models struggle to generalize to due to domain shift and limited training data. While this makes PBR-SR better suited for offline content creation (e.g., game asset preparation or digital twin generation), real-time deployment could benefit from more efficient alternatives.

2. Limitation of learning-based methods, or if the design can be improved

While our current learning-based baseline (CAMixerSR fine-tuned on our PBR dataset) achieves only modest improvements, we do not attribute this to an inherent limitation of learning-based methods. Rather, we believe that two key factors limit their current performance:

• Data limitation: The lack of large-scale, diverse PBR texture datasets paired with 3D mesh geometry under varying lighting conditions makes it difficult for learning-based methods to generalize effectively. Our current fine-tuning is performed on relatively limited PBR data, which likely restricts performance.
• Loss design: Existing loss functions in image space (e.g., pixel-wise or perceptual losses) do not adequately capture physically-based rendering characteristics, such as relighting consistency or material-specific shading effects, especially under novel views or illumination.

Looking ahead, we believe our optimization-based framework provides valuable insights for improving learning-based methods. Specifically, it opens up the possibility of incorporating physically motivated constraints, such as differentiable rendering loss and PBR-specific regularization, into the training pipeline. For example, when only PBR textures are available, 2D supervision can be used as a proxy. When PBR texture-mesh pairs are available, our differentiable rendering loss can be integrated into the training loop to guide learning with richer, geometry-aware supervision. We see this as a promising future direction to combine the efficiency of feed-forward inference with the fidelity of physically-grounded supervision.

3. Trade-offs between learning- and optimization-based approaches

Learning-based methods (e.g., CAMixerSR-FT) offer fast inference but rely on large-scale paired data and are typically supervised in texture space. As discussed in Lines 255–268, this can lead to distortions or a lack of sharpness in rendered appearance under novel lighting, since rendering fidelity is not explicitly optimized.

In contrast, our optimization-based approach is data-efficient and directly minimizes rendering-space error via differentiable rendering, leading to better relighting consistency, though at the cost of slower optimization.

As shown in Table 1, our method outperforms CAMixerSR-FT across all PBR channels on tested artist-created meshes:

• Albedo: 27.800 → 29.731
• Roughness: 30.642 → 31.602
• Metallic: 28.961 → 31.889
• Normal: 25.655 → 29.088

These results confirm the advantage of rendering-aware supervision. Figure 3 further shows that while CAMixerSR-FT produces plausible textures, it often fails to preserve material fidelity under relighting, unlike our method.

Similarly, we additionally tested on 32 PBR meshes that are AI-generated, and our method still outperforms CAMixerSR-FT across all PBR channels:

• Albedo: 31.439 → 33.821
• Roughness: 33.195 → 35.059
• Metallic: 27.583 → 35.352
• Normal: 31.054 → 34.564

We emphasize that our work fills a current gap in this space by proposing an optimization framework that leverages 2D priors while ensuring high rendering fidelity, even without additional PBR training data. At the same time, we believe future work could explore hybrid strategies, such as:

• Using learning-based models (e.g., CAMixerSR) for efficient initialization;
• Incorporating rendering-aware loss functions into the training of feed-forward networks;
• Jointly training on both texture-only and mesh-texture paired data, using our differentiable supervision as a guiding signal.

We have expanded our discussion accordingly in the revised manuscript.

We appreciate the detailed and valuable feedback from the reviewer, and we are glad that our method was recognized as addressing a 'critical industrial demand' with a 'novel distillation approach'. We also appreciate that our work was found to be 'well written', 'easy to follow', and 'simple to implement', while demonstrating 'good super-resolution results'. We respond to each of the points raised below.

1. Clarification on robust pixel-wise loss (weighting map resolution)

As shown in Table 2 of the main manuscript, we verify the overall effectiveness of this loss. To further analyze the impact of resolution, we conducted experiments using different weighting map sizes (8 to 256) on the ShoulderStrap mesh. The results are summarized below:

We observe that performance improves steadily from 8 to 64, with diminishing returns beyond 128. This suggests that moderate-resolution maps (e.g., 64 or 128) are sufficient to capture useful spatial inconsistencies introduced by our SR model. Higher resolutions offer marginal gains but incur significantly more memory overhead, as a separate weight map is optimized and stored for each rendering view. Hence, we adopt a resolution of 64 as a practical balance between quality and efficiency. These clarifications have been added to the revised manuscript.

2. Choice of lighting conditions

We utilize a single environment map in our experiments, as stated in the manuscript (line 220). Further experiments with varying and multiple environment maps demonstrated that the results were robust under moderate lighting changes, primarily due to constraints imposed by LR PBR maps, rendering the method insensitive to variations in illumination during optimization. This clarification has been added to the manuscript.

3. Evaluation scale and AI-generated meshes

To address the reviewer’s concern and further validate generalizability, we have added 32 AI-generated PBR-textured meshes sourced from the Hyper3D commercial platform. These meshes contain procedurally or AI-generated PBR textures with greater variety and realism, and each PBR channel is provided at 512×512 resolution. We apply 4× super-resolution to generate 2048×2048 outputs. The experimental protocol remains consistent with that used in the main 16-mesh evaluation. We present preliminary results comparing our method with the strongest baselines. Final results will be included in the revised manuscript, along with comparisons to other baselines and additional quantitative comparisons.

These additional evaluations on AI-generated PBR textures reinforce the general applicability and robustness of our method, beyond handcrafted or artist-designed assets.

4. Impact of different super-resolution models

We provide the analysis of the impact of different SR models in Appendix Section B Ablation on Image SR Models in PBR-SR, along with Table 4, where we compare three different SR models (DiffBIR, CAMixerSR, and StableSR) used for both initialization and rendering-time SR.

Our results show that DiffBIR consistently achieves the best performance across most PBR channels, including Albedo, Roughness, and Normal, as well as in final rendering PSNR. It outperforms CAMixerSR and StableSR by a notable margin in terms of both texture fidelity and perceptual rendering quality. While StableSR performs well on the Metallic channel, it underperforms on others and leads to lower overall rendering quality.

Importantly, our method is not tied to DiffBIR—it supports plug-and-play integration of other SR models. This flexibility is a key strength of our pipeline. However, we adopt DiffBIR as the default due to its strong balance of performance and efficiency: it not only yields high-quality outputs but also offers faster inference than StableSR, which is especially valuable in our iterative optimization process. We have clarified this point in the revised manuscript to emphasize the generality of our method with respect to different SR backbones.

5. Implementation details of Paint-it SR baseline

In Section 4.2 and Appendix Section A.3, we retain the core convolutional architecture and optimization framework of Paint-it, but make the following key changes:

• We replace the original text-to-image diffusion model and Score Distillation Sampling (SDS) loss with an image super-resolution model and a pixel-wise render loss.
• The deep convolutional block is re-initialized to output zero initially, allowing it to learn only the residual components over interpolated LR PBR maps.
• This residual learning setup allows the model to refine details while preserving the low-frequency information already present in the LR maps. While Paint-it SR shares a similar optimization framework, it is explicitly re-purposed for SR rather than text-to-texture synthesis. These modifications make it a strong baseline, yet our proposed method still outperforms it due to a more effective integration of SR priors and view-consistent optimization.

We have clarified the above points in the revised manuscript.

We appreciate the thoughtful review. We are pleased that our method was found to be 'well-motivated' and 'practical', and that both the effectiveness of our design and the clarity of our presentation were appreciated. We provide below a point-by-point response to each concern.

1. Expanded evaluation

To further strengthen the evaluation, we have added 32 additional meshes from the commercial Hyper3D website, which features AI-generated PBR textures with more diverse content. Each sample includes PBR maps at 512×512 resolution as input, and 2048×2048 resolution as GT. We perform 4× SR as in the main paper. All experimental settings remain consistent with those used on the original 16 meshes.

Below we provide preliminary results comparing our method against the two strongest baselines. Results from other baselines and visualizations will be added in the final version. Our method consistently outperforms both baselines across all PBR channels and in rendering quality.

Importantly, the trends observed on the newly added AI-generated meshes are consistent with those on the original 16 high-quality test cases. Across both evaluation sets, our method achieves robust improvements in all PBR texture channels and final rendered appearance, further confirming its generalization ability across diverse mesh sources and content types.

We will include the complete benchmark results and extended comparisons in the final manuscript.

2. Computational analysis

We provide a detailed discussion in Appendix Section A.2, where we report runtime across different resolutions:

• On a single NVIDIA A6000 GPU, optimization takes approximately 30 minutes for 2K-to-8K resolution.
• For 1K-to-2K resolution, the process takes less than 8 minutes on average. As our method is the first to tackle mesh PBR texture super-resolution without requiring external PBR texture datasets, we adopted an optimization-based approach that requires iterative updates. This design allows us to flexibly enhance high-resolution materials for arbitrary 3D assets without retraining or domain-specific data.

Our method is intended for offline SR of mesh PBR textures, and the optimized outputs can be directly loaded for real-time rendering in downstream applications. While the current optimization time is acceptable for offline use, we agree that improving efficiency is a valuable direction for future work.

3. Robustness analysis on degraded LR textures

To assess the robustness of our optimization-based method under degraded inputs, we conducted additional experiments with two common types of degradation:

• Gaussian noise added to the LR albedo map, with σ = 5, 10, 15, 20 (in 0–255 range);
• JPEG compression artifacts to the LR albedo map, with quality factors = 80, 60, 40, 20, 10 (lower values indicate stronger compression). We report both the final PSNR after optimization and the PSNR improvement from initialization (i.e., before vs. after optimization), for both the Albedo map and the rendered appearance (average over views) on the TableClock mesh.

• Our method consistently improves both Albedo and rendering quality, even under moderate or even strong degradation.
• Performance gradually decreases as degradation increases, which is expected, but the optimization process still provides meaningful improvements over the initialization.
• Notably, our method remains relatively robust to JPEG compression, showing stable performance even at high compression levels. These results confirm that while input quality does influence the final outcome, our method can tolerate a reasonable range of degradation and still benefit from the optimization process. We will include these findings in the supplementary material.

4. Dependency on pretrained SR models and material diversity

The core objective of our work is to investigate how to robustly enhance PBR texture maps by leveraging natural image priors, without requiring large-scale PBR texture datasets.

To mitigate domain gap issues, we introduce a rendering SR module that guides the optimization through rendered supervision and uses LR PBR maps as structural constraints across channels. This design allows our method to adapt and refine outputs for diverse material types without retraining.

Our test set includes a wide variety of material types (e.g., metals, plastics, woods, fabrics), and results in Table 1 show consistent improvements across all PBR channels, demonstrating the effectiveness of our strategy. Additionally, we conducted an ablation study using multiple pretrained 2D image SR models (DiffBIR, StableSR, and CAMixerSR), as shown in Appendix Section B: "Ablation on Image SR Models in PBR-SR" and Table 4. While DiffBIR yields the best overall performance, all models contribute to noticeable improvements in PBR texture quality, confirming the robustness of our pipeline regardless of SR model choice.

5. Discussion on failure cases/modes

Our model is generally able to generate HR PBR textures based on the provided LR PBR inputs. The main failure cases we observed occur when the input LR PBR maps are of extremely poor quality, lacking sufficient structural or textural clues. In such scenarios, the model struggles to infer plausible high-frequency details, which leads to suboptimal outputs. We have included representative failure case examples in the supplementary material to provide a clearer view of the method’s current limitations.

We have further clarified this point in the revised manuscript to emphasize generalizability across material types.

We greatly appreciate the detailed and valuable feedback from the reviewer, and we are glad that our method was found to be effective for PBR super resolution, yielding 'convincing results' with 'better quality and sharpness' than other baselines. We respond to each of the points raised below.

Weaknesses

1. Supplementary material clarity
We will clarify the supplementary material by revising the main text to explicitly outline the contents, including detailed experimental setups, computational requirements, and additional qualitative and quantitative results.

2. Notation (k, K), F_0, and L
We have improved the clarity by adopting standard notations:

• The specular reflectance is now denoted as $F_0$.
• The symbols previously used for diffuse and specular components ($L$) have been changed to $D$ and $S$, respectively, to avoid confusion with $L$, which continues to represent illumination.
• The notation $(k, K)$is now defined contextually or replaced with more intuitive expressions.

These changes have been reflected throughout the manuscript, including the equations and figure annotations. We believe these revisions will help make the paper more accessible to readers familiar with standard PBR terminology.

3. BRDF definition clarification We have now explicitly clarified the relationship between the specular reflection term and the Fresnel term to better support readers unfamiliar with microfacet-based BRDFs.

4. Notation of regularizer
To avoid confusion, the weighting regularizer is now explicitly denoted as $W$, clearly separating it from the rendering function $R^M$. This improves readability and interpretability.

5. Figure 2 clarity and completeness
We revised Figure 2 by simplifying the layout, explicitly including Ambient Occlusion (AO), and clearly labeling key components such as LR, HR, SR, $I^{HR}$, and $I^{SR}$.

6. Equation 6 consistency
We have corrected Equation 6 for consistent notation throughout. The ambiguous phrase “corresponding L1 texture input” has also been revised to clearly state “the corresponding L1 loss applied between textures.”

7. Contribution of extensive experiments
We have restructured the contribution section to focus on two core contributions. The extensive experiments are now described in the introduction to provide context, rather than being listed as a standalone contribution.

8. Table 1 optimization runtime
We now have included average runtime for our optimization method, and have clearly annotated runtimes for other optimization-based baselines in Table 1.

• Paint-it SR optimization time: around 30 minutes with 2K-to-8K resolution and less than 8 minutes with 1K-to-2K. Similar optimization time as PBR-SR (Ours).
• CAMixerSR-FT fine-tuning time: 9.5 hours of training.

---

Questions

1. Rendering approach
We use Nvdiffrast as our differentiable renderer (line 217), and a single environment map for illumination (line 220) during the default optimization process. The environment lighting is not randomly sampled, and our default setting uses one fixed HDR environment map. We will release the code and related data.

To assess robustness, we have tested with over five different environment maps. Each environment map is used individually, and the results remain consistent, as long as extreme lighting (e.g., pitch-black or overly bright) is avoided.

We have further tested the training with 2 or 3 different environment maps per sample (i.e., multiple renderings per step). Even with the increased diversity, the performance is still stable.

These results suggest that our method is robust to moderate lighting variation, thanks to the strong guidance from the LR PBR maps. This has been clarified in the revised manuscript.

2. Naive baseline using pretrained SR fine-tuning
In our paper, we compare three types of pipelines:

• (1) Direct inference using pretrained 2D SR models on individual maps.
• (2) Fine-tuning a 2D SR model on low-resolution PBR texture data.
• (3) Our optimization, which does not rely on any external PBR dataset and optimizes only on the given textured mesh.

Our method is, to the best of our knowledge, the first to address this problem setting, without any task-specific training, relying only on 2D natural image SR priors. This allows us to support arbitrary shapes and materials without requiring curated PBR datasets.

We agree that your proposed approach, pretraining an SR model on texture data and using it to initialize our optimization per channel, could be a promising extension that better leverages available data. It offers a good trade-off between generalization and instance-specific refinement. We see this as a valuable direction for future work. We have clarified this discussion in the revised manuscript.

3. Interpolation method for normal and metalness maps

We appreciate the reviewer raising this point. We conducted experiments to compare different interpolation methods. Below are the results on the Globe mesh:

• Metalness and Roughness maps: Bicubic performs slightly better than nearest-neighbor, mainly because of smoother transitions that help optimization, even for binary-valued maps.
• Normal maps: Bicubic interpolation without normalization provides better performance. Interestingly, direct bicubic interpolation yields slightly better performance (i.e., PSNR 34.299 vs. 32.855). Upon investigation, we found that publicly available normal maps often have pixel vectors with norms slightly less than 1 due to compression and quantization artifacts. Normalizing after interpolation may amplify artifacts and degrade quality.

Overall, bicubic interpolation serves as a robust and effective initialization, and any minor issues are corrected through our optimization process. These findings have been discussed in the manuscript.

4. Supplementary material corrections

• The incorrect reference at line 69 ("Fig. X") has been fixed.
• In Table 3, "w/o robust" refers to using pixel-wise loss without the robust weighting term. Other losses are still applied. This has been clarified in the Appendix (lines 63-75).

Limitations

We have added failure case examples in the supplementary material, including cases with poor LR texture inputs, to give a full picture of the method’s limitations.

Formatting and Ethical Concerns

We have revised the bibliography for consistent formatting and will remove potentially identifying elements such as voice-over mentions in the future.

We have clarified the above points in the revised manuscript.

Dear Reviewer,

Please respond to the rebuttal from the authors. Please note that, this year it is not allowed to finalize without discussion with authors.

Best,

Your AC