Deep Learning

Today’s deep neural networks have already become fully capable, especially in the vision field where they have approached or even surpassed human visual ability, and behind these deep learning networks there is always a basic framework, namely the convolutional neural network (CNN). CNN is mainly applied in the field of computer vision, so here I will briefly introduce the classic CNN model AlexNet. AlexNet was proposed by Alex Krizhevsky et al. in 2012. Although the model is rather rare nowadays (most people use resnet or something similar), it performed very well in the object detection field at the time and laid the foundation for other later CNN frameworks. ...

Optimization is a very difficult problem; usually one has to carefully design an objective function and related constraints, and also make sure the problem to be optimized is a convex function, but most real-world problems are non-convex. For non-convex optimization problems there are some basic optimization methods: SGD, Momentum, AdaGrad, RMSProp, Adam and so on Stochastic Gradient Descent (SGD) // stochastic gradient descent Given the dataset $X=\\{x^{(1)},x^{(2)},...,x^{(n)}\\}$ and the dataset labels $Y=\\{y^{(1)}, y^{(2)},...,y^{(n)}\\}$ ...