Deep Learning

Object recognition is much more complex than plain image classification: the algorithm model has to identify which objects are where in the image and draw boxes around them. Ross Girshick proposed an algorithm model that uses a convolutional neural network together with a support vector machine for object detection. Below is a brief analysis of this classic model. Basic structure The first problem is object localization, which the authors solve through the region recognition approach, using $selective \ search$. The second problem is the scarcity of training data; the paper proposes supervised pre-training on a large auxiliary dataset, followed by fine-tuning of specific parts on a small dataset. The experimental results also confirm that this approach is effective. The rough structure is as follows: ...

First of all, happy Teachers’ Day to all teachers, and thank you for your spirit of giving roses to others with a lingering fragrance on your hands. Then, after almost a month, I have rolled back to update the blog again; before that I was busy with joining the company, hoping to fit into the company’s development work as soon as possible, so I was “busy” for a month. Next let us continue with the introduction of classic image recognition network models. In previous posts we already introduced the AlexNet and VGG networks, both of which pushed the depth of neural networks of that time to the extreme, and subsequent neural network models basically performed no better. This is because as the network gets deeper, during error backpropagation the weights of the earlier network layers basically do not get updated and stay around some fixed value. In 2015, Kaiming He proposed a shortcut connection that propagates the error to the earlier network layers, greatly improving network performance. Below is a brief analysis of the ResNet network. ...

Neural networks already have several normalization algorithms. They all look very similar and basically share a unified form. This post refers to the Zhihu column SIGAI’s article on normalization algorithms; the link is at the bottom of the post. The basic problem When using a neural network to learn from a dataset, because the sources of the data may differ considerably, the values of different attributes may also span very different ranges; all of this causes errors in how the data is treated and leads to poor learning by the network. To solve this problem we normalize the data, that is, we regularize data whose value ranges vary widely into the same scale, which is beneficial for the network’s learning. Up to now normalization has many variants, the common ones being Batch Normalization (BN), Layer Normalization (LN), Instance Normalization (IN) and Group Normalization (GN). In fact their processing is similar (subtract the mean, divide by the standard deviation, then apply a linear mapping); the difference lies in the dimension along which the statistics are computed. The concrete formula is as follows: ...

VGG (Visual Geometry Group) is another classic convolutional neural network model after AlexNet. It won second place in the 2014 ILSVRC competition (first place went to GoogleLeNet), and the variants used most often in work and personal projects are VGG16 and VGG19, where 16 and 19 denote the number of layers in the network. Following on from the AlexNet model discussed previously, this post analyzes the VGG model. Basic architecture The training input images are $224 \times 224$ RGB images, and the only preprocessing is subtracting the mean RGB value of the training set from every pixel. The convolution layers use a kernel size of $3 \times 3$ with stride 1, and some convolution kernels use a $1 \times 1$ size; the max pooling layers use a kernel size of $2 \times 2$ with stride 2. Of the three fully connected layers at the end, the first two have 4096 channels, followed by a $softmax$ layer; the hidden layer activation function is $ReLU$, and the LRN local response normalization used in AlexNet is not used. The detailed structure is shown in the table below: ...