2021 on Simplestory

The PointNet Series

Sat, 07 Aug 2021 00:00:00 +0000

PointNet, PointNet++ and Frustum PointNet. The commonly used 3D point cloud object detection.

PointNet

PointNet applies deep learning directly to point cloud data. Point cloud data is a subset of the points in Euclidean space, and has three properties: disorder, the spatial relationship between points, and invariance.

Disorder. Point cloud data is a set and is insensitive to the order of the data, which requires the model to be invariant to different permutations of the data. There are several approaches. The first is to sort the point cloud by some rule, but for high-dimensional space it is very hard to find a stable sorting rule. Another is to treat it as a sequence and train it with an RNN, but this approach struggles to handle input elements of extremely large length (thousands to tens of thousands, as in a point cloud). Finally the authors use a symmetric function to aggregate the information of each point: $f(\lbrace x_1,x_2,\cdots,x_n\rbrace)\approx g(h(x_1),h(h(x_2),\cdots,h(x_n))$, where the left side of the equation is the goal and the right side is the designed symmetric function. From this we can see that each point in the point cloud is processed separately with $h$ and then passed into the symmetric function $g$. In the paper the authors choose $h$ to be an MLP and $g$ to be max pooling.
The spatial relationship between points. An object usually consists of point clouds in a specific space, and there are spatial relationships between points; PointNet exploits these relationships by concatenating global features with local features.
Invariance. The object represented by point cloud data should be invariant to some spatial transformations (rigid transformations). PointNet aligns the point cloud data before extracting features to guarantee invariance. The alignment is achieved by training a small network (T-Net) to obtain a transformation matrix and multiplying it with the input point cloud data. This operation can be seen as adjusting the spatial points; intuitively it rotates them into an orientation that is favorable for classification or segmentation (a rigid transformation). In the actual implementation a related loss function is also added to make the transformation matrix approximately orthogonal, so as to reduce the loss of point cloud information. (Regularization: $L_{reg}=\lVert I-AA^T\rVert_F^2$, that is, an L2 loss)

The rough structure of PointNet is as follows:

Multi-View 3D Object Detection Network

Fri, 06 Aug 2021 00:00:00 +0000

MV3D

SORT and DeepSORT

Thu, 05 Aug 2021 00:00:00 +0000

The multi-object tracking frameworks commonly used in industry at present.

SORT

This algorithm is based on an object detection network (Faster Rcnn) and makes use of the Kalman filter and the Hungarian algorithm (or the KM algorithm), greatly improving the speed of multi-object tracking. The rough pipeline is as follows:

The object detection algorithm obtains the object boxes Detections of the current frame, the Kalman filter obtains the Tracks of the current frame, and IOU matching is performed between the Detections and the Tracks. The final results fall into three categories:

Dynamic Head

Wed, 04 Aug 2021 00:00:00 +0000

Dynamic network head

Dynamic Convolution and Dynamic ReLU

Tue, 03 Aug 2021 00:00:00 +0000

Make it dynamic.

Dynamic Convolution

Ordinary static convolution applies the same kernel to all input images, whereas dynamic convolution adapts to different images and processes them with more suitable convolution parameters, which increases the expressive power of the model without increasing network depth or width.

Dynamic convolution dynamically aggregates multiple parallel convolution kernels according to attention. The attention dynamically adjusts the weight of each kernel according to the input, thereby producing an adaptive dynamic convolution. Because the attention is a function of the input, dynamic convolution is no longer a linear function. Superimposing kernels in a nonlinear way through attention gives stronger representational power. The rough structure is as follows:

CRF and Its Application in Segmentation

Sun, 16 May 2021 00:00:00 +0000

Conditional random field

Basic Networks for Image Segmentation

Fri, 30 Apr 2021 00:00:00 +0000

Image segmentation is pixel-level classification of an image, and mainly includes semantic segmentation and instance segmentation. Semantic segmentation only requires segmentation at the pixel level, while instance segmentation additionally requires separating different individuals on top of semantic segmentation. This post briefly introduces several basic semantic segmentation networks.

FCN

FCN (Full Convolution Network) is a classic image segmentation network whose main structure is an encoder-decoder, where the encoder extracts image feature information and the decoder interprets the extracted features. FCN replaces the fully connected layers in the network with convolution layers, so that 2D feature maps are obtained; these are then upsampled through deconvolution (transposed convolution), and finally a softmax layer produces the mask for each class. It can also handle input images of different sizes. The rough structure is shown below.