Bayesian decision theory considers how to choose the optimal class label based on the relevant probabilities and misclassification losses Suppose there are N possible class labels, that is $\mathbf{y}=\\{c_1,c_2,...,c_N\\}$, and $\lambda_{ij}$ is the loss incurred by misclassifying a sample whose true label is $c_j$ as $c_i$. Based on the posterior probability $P(c_i\vert\mathbf{x})$, we can obtain the expected loss incurred by classifying sample $\mathbf{x}$ as $c_i$, that is, the “conditional risk” on sample $\mathbf{x}$: ...

SVM is a very important model in machine learning, proposed by Vladimir Vapnik, the father of statistical learning theory This post takes binary classification with SVM as the example Derivation Given the training dataset $D=\\{(\mathbf{x_1},y_1),(\mathbf{x_2},y_2),...,(\mathbf{x_m},y_m)\\}, y_i \in \\{-1, +1\\}$, the most basic idea of a classifier is to find in the sample space a dividing space based on the training set D that separates the samples of different classes. The dividing hyperplane can be described by the following linear equation: ...

I have not updated the blog for a long time, having paused for about half a year, and I have also read half a year’s worth of books on machine learning and deep learning; now it is time to write some output Linear regression Given a dataset $D=\\{(\mathbf{x_1},y_1),(\mathbf{x_2},y_2),...,(\mathbf{x_m},y_m)\\}$, where $\mathbf{x_i}=(x_{i1};x_{i2};...;x_{id}), \ y_{i} \in R$. Linear regression tries to learn a linear model $$f(x_i)=wx_i+b$$such that $f(x_i)\simeq y_i$. Derivation Consider using MSE (mean squared error) to measure how good the model is; then the loss function $L(w,b)$ is: ...

G++ is the C++ compiler under Unix/Linux, and gdb is a powerful debugging tool under Unix/Linux. Most of the time, the C/C++ code we write does not run on Windows but on a Linux system. Here I record some usage of g++ and gdb in case they are needed. Using g++ When g++ performs compilation work, it needs 4 steps in total: Preprocessing, generating a .i file, mainly expanding header files and macro definitions: g++ -E test.cpp -o test.i Compilation, converting the preprocessed file into assembly language and generating a .s file: g++ -S test.i -o test.s Assembly, turning the assembly into object code (machine code) and generating a .o file, mainly converting the .s file into a binary file: g++ -c test.s -o test.o Linking, linking the object code to generate an executable program: g++ test.o -o test The steps above can also be done with a single command: ...