Deep Learning

Monocular 3D detection is an important and challenging task in the autonomous driving field; the current difficulty is how to accurately predict the depth of objects. MonoRCNN uses geometric relations to decouple the depth estimation problem into the object’s real height and its projected height in the image. The authors impose two conditions on the use of this model: The model only predicts the heading angle of the object, that is, the yaw angle The camera intrinsics are known during both training and inference Architecture ...

A lightweight stereo matching module is introduced into the 2D real-time object detection framework yolo to achieve efficient 3D object detection. The rough structure is shown below: Anchors For 3D object detection, the original yolo anchors designed for 2D objects need to be redesigned. The authors use 12 parameters to represent an anchor: $[x_{2d}, y_{2d}, w_{2d}, h_{2d}]$, representing the 2D object box $[c_x, c_y, z]$ represents the position of the 3D object center in the left image $[w_{3d}, h_{3d}, l_{3d}]$ represents the width, height and length of the 3D object box $[sin(2\alpha), cos(2\alpha)]$ to estimate the angle of the object Here the authors found that $[sin\alpha, cos\alpha]$ and $[sin(\alpha+\pi), cos(\alpha+\pi)]$ may produce identical rectangular boxes, so they use $[sin(2\alpha), cos(2\alpha)]$ to represent the angle of the object, and additionally add a classification branch to predict whether $\vert\alpha\vert \gt \frac{\pi}{2}$, which also indicates whether the object is facing the camera. ...

Point clouds are sparse in distribution, irregular in structure and unordered geometrically, so applying convolution operations directly on point clouds may make the generated features deviate greatly from their original representation. VoxelNet This model voxelizes the point cloud and then uses 3D convolution to obtain features. Specifically: The point cloud space is divided equally into a voxel grid of size $v_D$, $v_H$, $v_W$, and at most $T$ points are randomly kept within each cell (extras are removed, fewer are left as they are). A VFE layer (voxel feature encoding layer) is applied to each voxel cell to extract voxel features, which are merged into a 4D tensor. 3D convolution is applied to the tensor obtained above to extract features further, and then an RPN structure is used to obtain the results As shown below: ...

This network performs 3D object detection using stereo vision, and belongs to the image-based result-lifting methods in 3D detection, which decompose 3D detection into two parts: 2D object detection and depth estimation. Stereo RCNN extends Faster RCNN, as shown below: Backbone The authors use ResNet-101 combined with FPN, with shared weights. This backbone processes the left and right images at the same time, and the results are then merged as the input of Stereo RPN. ...