NeISF++

Abstract

Recent inverse rendering methods have greatly improved shape, material, and illumination reconstruction by utilizing polarization cues. However, existing methods only support dielectrics, ignoring conductors that are found everywhere in life. Since conductors and dielectrics have different reflection properties, using previous dielectrics-based methods will lead to obvious errors. In addition, conductors are glossy, which may cause strong specular reflection and is hard to reconstruct. To solve the above issues, we propose NeISF++, an inverse rendering pipeline that supports conductors and dielectrics. The key ingredient for our proposal is a general pBRDF that describes both conductors and dielectrics. As for the strong specular reflection problem, we propose a novel geometry initialization method using DoLP images. This physical cue is invariant to intensities and thus robust to strong specular reflections. Experimental results on our synthetic and real datasets show that our method surpasses the existing polarized inverse rendering methods for geometry and material decomposition as well as downstream tasks like relighting.

Mesh comparisons

Here we display side-by-side videos comparing our method to PISR.

Select a scene below:

Relighting comparisons

Here we display side-by-side videos comparing our method to NeISF and GT across different objects.

Select a scene and a baseline method below:

Helmet
Stanford scan
Pan
Plate
Metallic plate

Interactive visualization. Hover or tap to move the split.

Surface normal comparisons

Here we display side-by-side comparing our method to top-performing baselines across different scenes.

Select a scene and a baseline method below:

BRDF parameter comparisons

Here we display side-by-side comparing our method to Ours-unpol, which is an ablated version that does not use polarization cues.

Select a scene and a BRDF parameter below:

BibTeX


        @InProceedings{Li_NeISFpp_CVPR2025,
          title={NeISF++: Neural Incident Stokes Field for Polarized Inverse Rendering of Conductors and Dielectrics},
          author={Li, Chenhao and Ono, Taishi and Uemori, Takeshi and Nitta, Sho and Mihara, Hajime and Gatto, Alexander and Nagahara, Hajime and Moriuchi, Yusuke},
          booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
          month={June},
          year={2025},
          pages={26493-26503}
        }