Deciding which gt to assign to which anchor is an important problem in object detection, namely the label assignment problem. OTA offers a new angle on this problem: treating label assignment as an optimal transport problem. Label Assignment Previous label assignment strategies are mainly divided into two kinds: fixed and dynamic. For fixed label assignment, Anchor-based detection models usually use an IoU threshold as the assignment criterion (such as Fast-RCNN, Faster-RCNN, YoLo3, RetinaNet and so on), while some Anchor-free detection models designate anchors within a certain range of the object center as positive samples (such as FCOS, Foveabox and so on). Fixed assignment methods usually adopt a fixed assignment strategy (for example a fixed IoU threshold or a fixed radius), which is the same for objects of different sizes and shapes, so the model may end up with a suboptimal assignment. ...

PointNet, PointNet++ and Frustum PointNet. The commonly used 3D point cloud object detection. PointNet PointNet applies deep learning directly to point cloud data. Point cloud data is a subset of the points in Euclidean space, and has three properties: disorder, the spatial relationship between points, and invariance. Disorder. Point cloud data is a set and is insensitive to the order of the data, which requires the model to be invariant to different permutations of the data. There are several approaches. The first is to sort the point cloud by some rule, but for high-dimensional space it is very hard to find a stable sorting rule. Another is to treat it as a sequence and train it with an RNN, but this approach struggles to handle input elements of extremely large length (thousands to tens of thousands, as in a point cloud). Finally the authors use a symmetric function to aggregate the information of each point: $f(\lbrace x_1,x_2,\cdots,x_n\rbrace)\approx g(h(x_1),h(h(x_2),\cdots,h(x_n))$, where the left side of the equation is the goal and the right side is the designed symmetric function. From this we can see that each point in the point cloud is processed separately with $h$ and then passed into the symmetric function $g$. In the paper the authors choose $h$ to be an MLP and $g$ to be max pooling. The spatial relationship between points. An object usually consists of point clouds in a specific space, and there are spatial relationships between points; PointNet exploits these relationships by concatenating global features with local features. Invariance. The object represented by point cloud data should be invariant to some spatial transformations (rigid transformations). PointNet aligns the point cloud data before extracting features to guarantee invariance. The alignment is achieved by training a small network (T-Net) to obtain a transformation matrix and multiplying it with the input point cloud data. This operation can be seen as adjusting the spatial points; intuitively it rotates them into an orientation that is favorable for classification or segmentation (a rigid transformation). In the actual implementation a related loss function is also added to make the transformation matrix approximately orthogonal, so as to reduce the loss of point cloud information. (Regularization: $L_{reg}=\lVert I-AA^T\rVert_F^2$, that is, an L2 loss) The rough structure of PointNet is as follows: ...

3D point cloud detection using three views (bird’s eye view, front view, and the original RGB image). Bird’s view The point cloud data is divided equally into $M$ slices; each slice is gridded at a granularity of $0.1m$, and the highest point cloud height within each cell is extracted as the height map of the bird’s eye view. The intensity map uses the reflectance value of the point with the greatest height in the cell, and the density map counts the number of point cloud points $N$ in each cell and normalizes the feature as $\min (1,\frac{\log (N+1)}{\log 64})$, where 64 should be the number of laser beams. The final feature has $M+2$ channels. ...

The multi-object tracking frameworks commonly used in industry at present. SORT This algorithm is based on an object detection network (Faster Rcnn) and makes use of the Kalman filter and the Hungarian algorithm (or the KM algorithm), greatly improving the speed of multi-object tracking. The rough pipeline is as follows: The object detection algorithm obtains the object boxes Detections of the current frame, the Kalman filter obtains the Tracks of the current frame, and IOU matching is performed between the Detections and the Tracks. The final results fall into three categories: ...