Neural Radiance Fields for Novel-View Synthesis and Neural 3D Scene Representation: Advances in Quality, Scale, and Rendering Efficiency
Keywords:
volumetric rendering, computer graphics, Mip-NeRFAbstract
Neural scene representations rapidly transformed view synthesis and computer graphics following the introduction of Neural Radiance Fields. Rather than explicitly representing geometry with conventional meshes or point clouds, NeRF models optimize continuous neural functions that map spatial positions and viewing directions to volumetric density and emitted radiance. Combined with differentiable volume rendering, this formulation enables photorealistic novel views to be synthesized from collections of posed images. This review examines neural radiance-field research through 2022, focusing on improvements in image quality, scene scale, generalization, dynamic content, anti-aliasing, and rendering efficiency. The original NeRF formulation is reviewed together with positional encoding and related neural implicit representations. Mip-NeRF introduced integrated positional encoding to reduce aliasing, while NeRF-W addressed uncontrolled photographs containing appearance variation. PixelNeRF and IBRNet investigated generalization across scenes, and D-NeRF and Nerfies extended radiance fields to dynamic and deformable environments. By 2022, Mip-NeRF 360 expanded neural rendering to unbounded scenes, Mega-NeRF addressed large-scale environments, and Instant Neural Graphics Primitives dramatically reduced training time through multiresolution hash encoding. TensorRF and Plenoxels further demonstrated that explicit or factorized representations could offer substantial efficiency improvements compared with fully implicit multilayer perceptrons. The review analyzes rendering quality, training speed, memory consumption, scene representation, generalization, and scalability. Remaining challenges included sparse-view reconstruction, dynamic scenes, editing, geometry accuracy, relighting, and deployment on resource-constrained hardware.
References
Mildenhall B, Srinivasan PP, Tancik M, Barron JT, Ramamoorthi R, Ng R. NeRF: representing scenes as neural radiance fields for view synthesis. In: Vedaldi A, Bischof H, Brox T, Frahm JM, editors. Computer Vision—ECCV 2020. Cham: Springer; 2020. p. 405-421.
Sitzmann V, Zollhöfer M, Wetzstein G. Scene representation networks: continuous 3D-structure-aware neural scene representations. Adv Neural Inf Process Syst. 2019;32:1121-1132.
Lombardi S, Simon T, Saragih J, Schwartz G, Lehrmann A, Sheikh Y. Neural volumes: learning dynamic renderable volumes from images. ACM Trans Graph. 2019;38(4):65.
Tancik M, Srinivasan PP, Mildenhall B, Fridovich-Keil S, Raghavan N, Singhal U, et al. Fourier features let networks learn high frequency functions in low dimensional domains. Adv Neural Inf Process Syst. 2020;33:7537-7547.
Barron JT, Mildenhall B, Tancik M, Hedman P, Martin-Brualla R, Srinivasan PP. Mip-NeRF: a multiscale representation for anti-aliasing neural radiance fields. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021. p. 5855-5864.
Martin-Brualla R, Radwan N, Sajjadi MSM, Barron JT, Dosovitskiy A, Duckworth D. NeRF in the wild: neural radiance fields for unconstrained photo collections. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021. p. 7210-7219.
Pumarola A, Corona E, Pons-Moll G, Moreno-Noguer F. D-NeRF: neural radiance fields for dynamic scenes. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021. p. 10318-10327.
Park K, Sinha U, Barron JT, Bouaziz S, Goldman DB, Seitz SM, Martin-Brualla R. Nerfies: deformable neural radiance fields. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021. p. 5865-5874.
Yu A, Ye V, Tancik M, Kanazawa A. PixelNeRF: neural radiance fields from one or few images. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021. p. 4578-4587.
Wang Q, Wang Z, Genova K, Srinivasan PP, Zhou H, Barron JT, et al. IBRNet: learning multi-view image-based rendering. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021. p. 4690-4699.
Yu A, Fridovich-Keil S, Tancik M, Chen Q, Recht B, Kanazawa A. PlenOctrees for real-time rendering of neural radiance fields. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021. p. 5752-5761.
Liu L, Gu J, Zaw Lin K, Chua TS, Theobalt C. Neural sparse voxel fields. Adv Neural Inf Process Syst. 2020;33:15651-15663.
Müller T, Evans A, Schied C, Keller A. Instant neural graphics primitives with a multiresolution hash encoding. ACM Trans Graph. 2022;41(4):102.
Barron JT, Mildenhall B, Verbin D, Srinivasan PP, Hedman P. Mip-NeRF 360: unbounded anti-aliased neural radiance fields. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022. p. 5470-5479.
Turki H, Ramanan D, Satyanarayanan M. Mega-NeRF: scalable construction of large-scale NeRFs for virtual fly-throughs. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022. p. 12922-12931.
Verbin D, Hedman P, Mildenhall B, Zickler T, Barron JT, Srinivasan PP. Ref-NeRF: structured view-dependent appearance for neural radiance fields. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022. p. 5481-5490.
Chen A, Xu Z, Geiger A, Yu J, Su H. TensorF: tensorial radiance fields. In: Avidan S, Brostow G, Cissé M, Farinella GM, Hassner T, editors. Computer Vision—ECCV 2022. Cham: Springer; 2022. p. 333-350.
Fridovich-Keil S, Yu A, Tancik M, Chen Q, Recht B, Kanazawa A. Plenoxels: radiance fields without neural networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022. p. 5501-5510.
Reiser C, Peng S, Liao Y, Geiger A. KiloNeRF: speeding up neural radiance fields with thousands of tiny MLPs. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021. p. 14335-14345.
Schwarz K, Liao Y, Niemeyer M, Geiger A. GRAF: generative radiance fields for 3D-aware image synthesis. Adv Neural Inf Process Syst. 2020;33:20154-20166.