Object detection with Faster R-CNN

An object detector using region proposals and a pretrained VGG16 backbone.

Overlapping parked cars detected in a VOC2007 test image. View full size (new tab): Faster R-CNN predictions around five parked cars

Detector architecture

I implemented the region proposal network, RoI pooling, classification head, box regression, and the end-to-end loss around the course-provided pretrained VGG16 backbone. Each feature-map location uses nine anchors across three scales and three aspect ratios. At test time, non-maximum suppression filters duplicate boxes.

Training and evaluation

The detector trained on VOC2007 trainval, with 5,011 images, and was evaluated on the 4,952-image test split at IoU 0.5. After 25 epochs, it reached 0.6086 mean average precision across the 20 VOC classes. The curves show the four loss terms settling after the scheduled learning-rate reductions.

RPN and detection classification and localization losses from the 25-epoch run. View full size (new tab): Four Faster R-CNN training loss curves across 25 epochs

Detection examples

The retained checkpoint finds multiple instances in crowded test images, including five parked cars and a dining-table scene with bottles and people. These are raw assignment outputs after per-class non-maximum suppression.

Predictions for two bottles and three people at a dining table. View full size (new tab): Faster R-CNN predictions around people and bottles at a dining table

Why I kept the shorter run

A 100-epoch follow-up lowered the training classification loss but reduced test mAP from 0.6086 to 0.5693. I kept the 25-epoch checkpoint because the longer run generalized worse on the held-out test set.

The longer run achieved lower training loss while its test mAP fell. View full size (new tab): Comparison of Faster R-CNN classification loss in 25-epoch and 100-epoch runs