Neural Radiance Fields (NeRFs) can be dramatically accelerated by spatial grid representations. However, they do not explicitly reason about scale and so introduce aliasing artifacts when reconstructing scenes captured at different camera distances. Mip-NeRF and its extensions propose scale-aware renderers that project volumetric frustums rather than point samples but such approaches rely on positional encodings that are not readily compatible with grid methods. We propose a simple modification to grid-based models by training model heads at different spatial grid resolutions. At render time, we simply use coarser grids to render samples that cover larger volumes. Our method can be easily applied to existing accelerated NeRF methods and significantly improves rendering quality (reducing error rates by 20–90% across synthetic and unbounded real-world scenes) while incurring minimal performance overhead (as each model head is quick to evaluate). Compared to Mip-NeRF, we reduce error rates by 20% while training over 60× faster.
Overview
Approach
Most NeRF methods assume that training and test-time cameras capture scene content from a roughly constant distance:
They degrade and render blurry views in less constrained settings:
This is due to NeRF being scale-unaware, as it reasons about point samples instead of volumes. We address this by training a pyramid of NeRFs that divide the scene at different resolutions. We use "coarse" NeRFs for far-away samples, and finer NeRF for close-up samples:
Multiscale Blender
We evaluate PyNeRF against the Multiscale Blender dataset. We outperform other fast NeRF methods (such as K-Planes and Nerfacto) by a wide margin. We provide slightly better results than Mip-NeRF while training over 60× faster.
Blender-A
We provide additional scenes that test common aliasing patterns and compare to prior work. Our method provides the best reconstruction results.
Real-World Scenes
We also test our method on real-world captures such the Boat scene from the ADOP dataset. Here, we zoom into the boat following a novel camera trajectory that differs greatly from the training image views.
Concurrent Work
Please also check out Tri-MipRF and Mip-VoG, which use prefiltering-based approaches to handle anti-aliasing. Zip-NeRF uses multisampling to address the same problem.
Citation
@InProceedings{turki2023pynerf,
title = {PyNeRF: Pyramidal Neural Radiance Fields},
author = {Haithem Turki and Michael Zollh\"{o}fer and Christian Richardt and Deva Ramanan},
booktitle={Thirty-Seventh Conference on Neural Information Processing Systems},
year = {2023}
}
