Road Marker Reconstruct
3D point cloud reconstruction from road markers using four synchronized car cameras
3D Reconstruction from Road Marker Feature Points
This was the final project for the Computer Vision course, run in collaboration with ITRI (Industrial Technology Research Institute). Given synchronized video from four onboard cameras (front, front-left, front-right, back) on a driving car, the goal was to reconstruct a 3D point cloud of road markers for visual-based localization of an autonomous vehicle.
Dataset
The dataset from ITRI contained three public video sequences and two private test sequences. Each timestamp in a sequence included:
- Raw images from four cameras with known intrinsic and extrinsic parameters (provided as ROS tf2 transforms)
- Detected road marker bounding boxes with five classes: zebra crossing, stop line, arrow, junction box, and other
- Ego car masks per camera to filter out the vehicle itself from the image
- Initial pose and sub-map points for ICP-based evaluation
- Ground truth poses (for public sequences) to benchmark localization accuracy
Evaluation was performed using Iterative Closest Point (ICP): the predicted point cloud, together with the sub-map and initial pose, was aligned against the ground truth to compute a localization error.
Method
We proposed a corner-detection-based 3D reconstruction algorithm. The pipeline:
- Video Input — four synchronized camera streams with timestamps
- Image Preprocessing — extract road marker regions using the provided bounding boxes, then apply morphological processing (erosion/dilation) to clean up the masks
- Segmentation — convert to binary masks using HSV thresholding with simple mask filtering to separate road markers from the road surface, achieving automatic traffic signal segmentation
- Corner Detection — apply a corner detection algorithm on the segmentation result to extract feature points at road marker corners
- 3D Reconstruction — project the detected 2D corner points into 3D using the pinhole camera model with the known camera intrinsics and extrinsics
- Point Cloud Assembly — concatenate point clouds from four consecutive frames across all cameras for a denser reconstruction
We performed ablation studies across the three public sequences to determine optimal hyperparameters for the morphological operations and corner detection thresholds.
3D point cloud reconstructed from the four onboard cameras
Results
Our team placed 6th out of 18 teams on the Codalab competition leaderboard.
| Video 1 | Video 2 |
|---|---|
 |
 |
| Test Video 1 | Test Video 2 |
|---|---|
 |
 |