Posts

VGG (Visual Geometry Group) is another classic convolutional neural network model after AlexNet. It won second place in the 2014 ILSVRC competition (first place went to GoogleLeNet), and the variants used most often in work and personal projects are VGG16 and VGG19, where 16 and 19 denote the number of layers in the network. Following on from the AlexNet model discussed previously, this post analyzes the VGG model. Basic architecture The training input images are $224 \times 224$ RGB images, and the only preprocessing is subtracting the mean RGB value of the training set from every pixel. The convolution layers use a kernel size of $3 \times 3$ with stride 1, and some convolution kernels use a $1 \times 1$ size; the max pooling layers use a kernel size of $2 \times 2$ with stride 2. Of the three fully connected layers at the end, the first two have 4096 channels, followed by a $softmax$ layer; the hidden layer activation function is $ReLU$, and the LRN local response normalization used in AlexNet is not used. The detailed structure is shown in the table below: ...

Today’s deep neural networks have already become fully capable, especially in the vision field where they have approached or even surpassed human visual ability, and behind these deep learning networks there is always a basic framework, namely the convolutional neural network (CNN). CNN is mainly applied in the field of computer vision, so here I will briefly introduce the classic CNN model AlexNet. AlexNet was proposed by Alex Krizhevsky et al. in 2012. Although the model is rather rare nowadays (most people use resnet or something similar), it performed very well in the object detection field at the time and laid the foundation for other later CNN frameworks. ...

The previous post mentioned an efficient tree ensemble algorithm, the Xgboost model, but its efficiency is still low when dealing with data that has a large feature dimension and a large number of samples. Compared with models such as GBDT and Xgboost, the Lightgbm model open-sourced by Microsoft uses a variety of techniques to reduce the amount of data and the number of features, which greatly speeds up training while keeping accuracy. ...

Today it occurred to me that I have not updated my blog for a long time; lately I have been busy with written tests and interviews, and with my graduation project. Now that things have come to a pause today, I might as well organize the Xgboost model I reviewed a few days ago and post an article. Tree ensemble Ensemble algorithms are a broad and very important class of algorithms in machine learning, generally divided into Bagging and Boosting. Ensembling decision trees is of course a customary technique. For tree ensembles, the representative of the Bagging method is Random Forest, which randomly groups attributes and data for decision tree learning and finally aggregates the results; the representative of the Boosting method is AdaBoost and GBDT (gradient boosted decision trees). These algorithms are all quite important and cannot be summed up in a sentence or two, so I will not expand on them. Within tree Boosting, Tianqi Chen proposed a new model, Xgboost. This model makes use of the second derivative of the objective function and also adds a regularization term to prevent the model from overfitting. Below is an interpretation of the Xgboost model. ...