2021

PointNet, PointNet++ and Frustum PointNet. The commonly used 3D point cloud object detection. PointNet PointNet applies deep learning directly to point cloud data. Point cloud data is a subset of the points in Euclidean space, and has three properties: disorder, the spatial relationship between points, and invariance. Disorder. Point cloud data is a set and is insensitive to the order of the data, which requires the model to be invariant to different permutations of the data. There are several approaches. The first is to sort the point cloud by some rule, but for high-dimensional space it is very hard to find a stable sorting rule. Another is to treat it as a sequence and train it with an RNN, but this approach struggles to handle input elements of extremely large length (thousands to tens of thousands, as in a point cloud). Finally the authors use a symmetric function to aggregate the information of each point: $f(\lbrace x_1,x_2,\cdots,x_n\rbrace)\approx g(h(x_1),h(h(x_2),\cdots,h(x_n))$, where the left side of the equation is the goal and the right side is the designed symmetric function. From this we can see that each point in the point cloud is processed separately with $h$ and then passed into the symmetric function $g$. In the paper the authors choose $h$ to be an MLP and $g$ to be max pooling. The spatial relationship between points. An object usually consists of point clouds in a specific space, and there are spatial relationships between points; PointNet exploits these relationships by concatenating global features with local features. Invariance. The object represented by point cloud data should be invariant to some spatial transformations (rigid transformations). PointNet aligns the point cloud data before extracting features to guarantee invariance. The alignment is achieved by training a small network (T-Net) to obtain a transformation matrix and multiplying it with the input point cloud data. This operation can be seen as adjusting the spatial points; intuitively it rotates them into an orientation that is favorable for classification or segmentation (a rigid transformation). In the actual implementation a related loss function is also added to make the transformation matrix approximately orthogonal, so as to reduce the loss of point cloud information. (Regularization: $L_{reg}=\lVert I-AA^T\rVert_F^2$, that is, an L2 loss) The rough structure of PointNet is as follows: ...

3D point cloud detection using three views (bird’s eye view, front view, and the original RGB image). Bird’s view The point cloud data is divided equally into $M$ slices; each slice is gridded at a granularity of $0.1m$, and the highest point cloud height within each cell is extracted as the height map of the bird’s eye view. The intensity map uses the reflectance value of the point with the greatest height in the cell, and the density map counts the number of point cloud points $N$ in each cell and normalizes the feature as $\min (1,\frac{\log (N+1)}{\log 64})$, where 64 should be the number of laser beams. The final feature has $M+2$ channels. ...

The multi-object tracking frameworks commonly used in industry at present. SORT This algorithm is based on an object detection network (Faster Rcnn) and makes use of the Kalman filter and the Hungarian algorithm (or the KM algorithm), greatly improving the speed of multi-object tracking. The rough pipeline is as follows: The object detection algorithm obtains the object boxes Detections of the current frame, the Kalman filter obtains the Tracks of the current frame, and IOU matching is performed between the Detections and the Tracks. The final results fall into three categories: ...

Object detection has produced many algorithms because of the complexity of combining the localization and classification branches. These methods improve algorithm performance from different angles, but are hard to measure in a unified way. This paper uses multiple attention mechanisms to unify different object detection heads: attention over the feature level, over the spatial position and across channels, corresponding respectively to scale awareness, spatial awareness and task awareness. This method can noticeably improve model performance without increasing the computational cost. ...