2018

Optimization is a very difficult problem; usually one has to carefully design an objective function and related constraints, and also make sure the problem to be optimized is a convex function, but most real-world problems are non-convex. For non-convex optimization problems there are some basic optimization methods: SGD, Momentum, AdaGrad, RMSProp, Adam and so on Stochastic Gradient Descent (SGD) // stochastic gradient descent Given the dataset $X=\\{x^{(1)},x^{(2)},...,x^{(n)}\\}$ and the dataset labels $Y=\\{y^{(1)}, y^{(2)},...,y^{(n)}\\}$ ...

CTR (click-through rate) prediction plays an important role in content recommendation and delivery, and the FFM model is good at handling this kind of data. The FFM model is a variant of the FM model, or rather the FM model is a special case of the FFM model Rough pipeline Convert the dataset format (using the LIBSVM or FFMs style) Compute the loss function (using the exponential loss function) Optimize the parameters (using the AdaGrad + SG strategy) Converting the data format LIBSVM format (does not store zero-valued features): ...

Decision tree, a common machine learning method, is somewhat like a flowchart. A sample is divided into a set according to the value it takes on some feature, and this is done recursively Basic algorithm Input: training set $D=\\{(\mathbf{x_1},y_1),(\mathbf{x_2},y_2),...,(\mathbf{x_m},y_m)\\}$; attribute set $A=\\{a_1,a_2,...,a_d\\}$ Output: a decision tree with node as the root node Simple process: Using some splitting criterion, the optimal feature is selected to split the samples, and this is done recursively, finally yielding the branch nodes of the decision tree ...

This time I am putting together the commonly used scikit-learn packages as a memo K-nearest neighbors: from sklearn.neighbors import KNeighborsClassifier Linear regression: from sklearn.linear_model import LinearRegression Logistic regression: from sklearn.linear_model import LogisticRgression Decision tree: from sklearn.tree import DecisionTreeClassifier Naive Bayes: Gaussian distribution: from sklearn.neive_bayes import GaussianNB Multinomial distribution: from sklearn.neive_baybes import MultinomnalNB Bernoulli distribution: from sklearn.neive_baybes import BernoulliNB Support vector machine: Classification: from sklearn.svm import SVC Regression: from sklearn.svm import SVR Ensemble algorithms (sklearn.ensemble): bagging: Classification: from sklearn.ensemble import BaggingClassifier ...