2023 on Simplestory

LSS

Sat, 24 Jun 2023 00:00:00 +0000

Lift, Splat, Shoot

MonoRCNN

Fri, 23 Jun 2023 00:00:00 +0000

Monocular 3D detection is an important and challenging task in the autonomous driving field; the current difficulty is how to accurately predict the depth of objects. MonoRCNN uses geometric relations to decouple the depth estimation problem into the object’s real height and its projected height in the image.

The authors impose two conditions on the use of this model:

The model only predicts the heading angle of the object, that is, the yaw angle
The camera intrinsics are known during both training and inference

Architecture

YOLOStereo3D

Fri, 23 Jun 2023 00:00:00 +0000

A lightweight stereo matching module is introduced into the 2D real-time object detection framework yolo to achieve efficient 3D object detection. The rough structure is shown below:

Anchors

For 3D object detection, the original yolo anchors designed for 2D objects need to be redesigned. The authors use 12 parameters to represent an anchor:

$[x_{2d}, y_{2d}, w_{2d}, h_{2d}]$, representing the 2D object box
$[c_x, c_y, z]$ represents the position of the 3D object center in the left image
$[w_{3d}, h_{3d}, l_{3d}]$ represents the width, height and length of the 3D object box
$[sin(2\alpha), cos(2\alpha)]$ to estimate the angle of the object

Here the authors found that $[sin\alpha, cos\alpha]$ and $[sin(\alpha+\pi), cos(\alpha+\pi)]$ may produce identical rectangular boxes, so they use $[sin(2\alpha), cos(2\alpha)]$ to represent the angle of the object, and additionally add a classification branch to predict whether $\vert\alpha\vert \gt \frac{\pi}{2}$, which also indicates whether the object is facing the camera.

VoxelNet and SECOND

Sun, 05 Mar 2023 00:00:00 +0000

Point clouds are sparse in distribution, irregular in structure and unordered geometrically, so applying convolution operations directly on point clouds may make the generated features deviate greatly from their original representation.

VoxelNet

This model voxelizes the point cloud and then uses 3D convolution to obtain features. Specifically:

The point cloud space is divided equally into a voxel grid of size $v_D$, $v_H$, $v_W$, and at most $T$ points are randomly kept within each cell (extras are removed, fewer are left as they are).
A VFE layer (voxel feature encoding layer) is applied to each voxel cell to extract voxel features, which are merged into a 4D tensor.
3D convolution is applied to the tensor obtained above to extract features further, and then an RPN structure is used to obtain the results

As shown below:

Stereo RCNN

Fri, 27 Jan 2023 00:00:00 +0000

This network performs 3D object detection using stereo vision, and belongs to the image-based result-lifting methods in 3D detection, which decompose 3D detection into two parts: 2D object detection and depth estimation.

Stereo RCNN extends Faster RCNN, as shown below:

Backbone

The authors use ResNet-101 combined with FPN, with shared weights. This backbone processes the left and right images at the same time, and the results are then merged as the input of Stereo RPN.