Machine Learning

This time I am putting together the commonly used scikit-learn packages as a memo K-nearest neighbors: from sklearn.neighbors import KNeighborsClassifier Linear regression: from sklearn.linear_model import LinearRegression Logistic regression: from sklearn.linear_model import LogisticRgression Decision tree: from sklearn.tree import DecisionTreeClassifier Naive Bayes: Gaussian distribution: from sklearn.neive_bayes import GaussianNB Multinomial distribution: from sklearn.neive_baybes import MultinomnalNB Bernoulli distribution: from sklearn.neive_baybes import BernoulliNB Support vector machine: Classification: from sklearn.svm import SVC Regression: from sklearn.svm import SVR Ensemble algorithms (sklearn.ensemble): bagging: Classification: from sklearn.ensemble import BaggingClassifier ...

Bayesian decision theory considers how to choose the optimal class label based on the relevant probabilities and misclassification losses Suppose there are N possible class labels, that is $\mathbf{y}=\\{c_1,c_2,...,c_N\\}$, and $\lambda_{ij}$ is the loss incurred by misclassifying a sample whose true label is $c_j$ as $c_i$. Based on the posterior probability $P(c_i\vert\mathbf{x})$, we can obtain the expected loss incurred by classifying sample $\mathbf{x}$ as $c_i$, that is, the “conditional risk” on sample $\mathbf{x}$: ...

SVM is a very important model in machine learning, proposed by Vladimir Vapnik, the father of statistical learning theory This post takes binary classification with SVM as the example Derivation Given the training dataset $D=\\{(\mathbf{x_1},y_1),(\mathbf{x_2},y_2),...,(\mathbf{x_m},y_m)\\}, y_i \in \\{-1, +1\\}$, the most basic idea of a classifier is to find in the sample space a dividing space based on the training set D that separates the samples of different classes. The dividing hyperplane can be described by the following linear equation: ...

I have not updated the blog for a long time, having paused for about half a year, and I have also read half a year’s worth of books on machine learning and deep learning; now it is time to write some output Linear regression Given a dataset $D=\\{(\mathbf{x_1},y_1),(\mathbf{x_2},y_2),...,(\mathbf{x_m},y_m)\\}$, where $\mathbf{x_i}=(x_{i1};x_{i2};...;x_{id}), \ y_{i} \in R$. Linear regression tries to learn a linear model $$f(x_i)=wx_i+b$$such that $f(x_i)\simeq y_i$. Derivation Consider using MSE (mean squared error) to measure how good the model is; then the loss function $L(w,b)$ is: ...