2019

First of all, happy Teachers’ Day to all teachers, and thank you for your spirit of giving roses to others with a lingering fragrance on your hands. Then, after almost a month, I have rolled back to update the blog again; before that I was busy with joining the company, hoping to fit into the company’s development work as soon as possible, so I was “busy” for a month. Next let us continue with the introduction of classic image recognition network models. In previous posts we already introduced the AlexNet and VGG networks, both of which pushed the depth of neural networks of that time to the extreme, and subsequent neural network models basically performed no better. This is because as the network gets deeper, during error backpropagation the weights of the earlier network layers basically do not get updated and stay around some fixed value. In 2015, Kaiming He proposed a shortcut connection that propagates the error to the earlier network layers, greatly improving network performance. Below is a brief analysis of the ResNet network. ...

Neural networks already have several normalization algorithms. They all look very similar and basically share a unified form. This post refers to the Zhihu column SIGAI’s article on normalization algorithms; the link is at the bottom of the post. The basic problem When using a neural network to learn from a dataset, because the sources of the data may differ considerably, the values of different attributes may also span very different ranges; all of this causes errors in how the data is treated and leads to poor learning by the network. To solve this problem we normalize the data, that is, we regularize data whose value ranges vary widely into the same scale, which is beneficial for the network’s learning. Up to now normalization has many variants, the common ones being Batch Normalization (BN), Layer Normalization (LN), Instance Normalization (IN) and Group Normalization (GN). In fact their processing is similar (subtract the mean, divide by the standard deviation, then apply a linear mapping); the difference lies in the dimension along which the statistics are computed. The concrete formula is as follows: ...

VGG (Visual Geometry Group) is another classic convolutional neural network model after AlexNet. It won second place in the 2014 ILSVRC competition (first place went to GoogleLeNet), and the variants used most often in work and personal projects are VGG16 and VGG19, where 16 and 19 denote the number of layers in the network. Following on from the AlexNet model discussed previously, this post analyzes the VGG model. Basic architecture The training input images are $224 \times 224$ RGB images, and the only preprocessing is subtracting the mean RGB value of the training set from every pixel. The convolution layers use a kernel size of $3 \times 3$ with stride 1, and some convolution kernels use a $1 \times 1$ size; the max pooling layers use a kernel size of $2 \times 2$ with stride 2. Of the three fully connected layers at the end, the first two have 4096 channels, followed by a $softmax$ layer; the hidden layer activation function is $ReLU$, and the LRN local response normalization used in AlexNet is not used. The detailed structure is shown in the table below: ...

Today’s deep neural networks have already become fully capable, especially in the vision field where they have approached or even surpassed human visual ability, and behind these deep learning networks there is always a basic framework, namely the convolutional neural network (CNN). CNN is mainly applied in the field of computer vision, so here I will briefly introduce the classic CNN model AlexNet. AlexNet was proposed by Alex Krizhevsky et al. in 2012. Although the model is rather rare nowadays (most people use resnet or something similar), it performed very well in the object detection field at the time and laid the foundation for other later CNN frameworks. ...