Neural radiance fields

A PyTorch model that renders new views of a scene from images with known camera positions.

Views rendered by my NeRF model after 200,000 training steps, along a 360° camera path around the Lego scene.

Best held-out view, image 020: 32.66 dB PSNR. Mean PSNR across 25 held-out views: 30.71 dB. View full size (new tab): Best held-out Lego scene view, with NeRF render on the left and ground truth on the right

Scene and representation

For this COSC 83 assignment, I trained a NeRF on the synthetic Lego scene from the Blender dataset. The dataset has 100 training views, 13 validation views, and 25 held-out test views with known camera poses. I composited the source images over white and trained at 400 × 400 resolution to fit the run within the available GPU budget.

Rendering pipeline

The implementation combines positional encoding, a view-dependent MLP with a skip connection, camera-ray generation, stratified sampling, and differentiable volume rendering. A coarse network distributes extra samples along each ray, and a fine network renders the final color. Training used 64 coarse and 128 fine samples per ray for 200,000 iterations.

Training loss and per-batch PSNR. The batch metric is noisy because it is calculated from one random ray batch. View full size (new tab): Training MSE and per-batch PSNR across 200,000 NeRF iterations

Held-out views

I evaluated all 25 held-out camera poses. The mean test PSNR was 30.71 dB across 25 views; the best view reached 32.66 dB and the lowest was 27.57 dB. The best-view comparison shows the rendered image beside its ground truth.

Lowest-scoring held-out view, image 010: 27.57 dB PSNR; fine tread and edge detail are softer. View full size (new tab): Lowest-scoring held-out Lego scene view, with NeRF render on the left and ground truth on the right

Depth from the learned field

The same field also produced a disparity frame along a spiral camera path. Brighter regions are nearer to the camera, separating the bulldozer from the background in the recovered scene geometry.

Inverse-depth frame from the spiral render. Brighter values are closer to the camera. View full size (new tab): Grayscale NeRF disparity map of the Lego bulldozer scene