Posts on Simplestory

DDIM

Sat, 17 Aug 2024 00:00:00 +0000

Denoising Diffusion Implicit Models

DDPM

Sat, 17 Aug 2024 00:00:00 +0000

Denoising Diffusion Probabilistic Models

LDM

Sat, 17 Aug 2024 00:00:00 +0000

Latent Diffusion Models

GAN

Sun, 11 Aug 2024 00:00:00 +0000

Generative Adversarial Network

VAE

Sun, 11 Aug 2024 00:00:00 +0000

Varitional Autoencoder

VQ-GAN

Sun, 11 Aug 2024 00:00:00 +0000

Vector Quantized Generative Adversarial Network

VQ-VAE

Sun, 11 Aug 2024 00:00:00 +0000

Vector Quantised Variational AutoEncoder

LSS

Sat, 24 Jun 2023 00:00:00 +0000

Lift, Splat, Shoot

MonoRCNN

Fri, 23 Jun 2023 00:00:00 +0000

Monocular 3D detection is an important and challenging task in the autonomous driving field; the current difficulty is how to accurately predict the depth of objects. MonoRCNN uses geometric relations to decouple the depth estimation problem into the object’s real height and its projected height in the image.

The authors impose two conditions on the use of this model:

The model only predicts the heading angle of the object, that is, the yaw angle
The camera intrinsics are known during both training and inference

Architecture

YOLOStereo3D

Fri, 23 Jun 2023 00:00:00 +0000

A lightweight stereo matching module is introduced into the 2D real-time object detection framework yolo to achieve efficient 3D object detection. The rough structure is shown below:

Anchors

For 3D object detection, the original yolo anchors designed for 2D objects need to be redesigned. The authors use 12 parameters to represent an anchor:

$[x_{2d}, y_{2d}, w_{2d}, h_{2d}]$, representing the 2D object box
$[c_x, c_y, z]$ represents the position of the 3D object center in the left image
$[w_{3d}, h_{3d}, l_{3d}]$ represents the width, height and length of the 3D object box
$[sin(2\alpha), cos(2\alpha)]$ to estimate the angle of the object

Here the authors found that $[sin\alpha, cos\alpha]$ and $[sin(\alpha+\pi), cos(\alpha+\pi)]$ may produce identical rectangular boxes, so they use $[sin(2\alpha), cos(2\alpha)]$ to represent the angle of the object, and additionally add a classification branch to predict whether $\vert\alpha\vert \gt \frac{\pi}{2}$, which also indicates whether the object is facing the camera.

VoxelNet and SECOND

Sun, 05 Mar 2023 00:00:00 +0000

Point clouds are sparse in distribution, irregular in structure and unordered geometrically, so applying convolution operations directly on point clouds may make the generated features deviate greatly from their original representation.

VoxelNet

This model voxelizes the point cloud and then uses 3D convolution to obtain features. Specifically:

The point cloud space is divided equally into a voxel grid of size $v_D$, $v_H$, $v_W$, and at most $T$ points are randomly kept within each cell (extras are removed, fewer are left as they are).
A VFE layer (voxel feature encoding layer) is applied to each voxel cell to extract voxel features, which are merged into a 4D tensor.
3D convolution is applied to the tensor obtained above to extract features further, and then an RPN structure is used to obtain the results

As shown below:

Stereo RCNN

Fri, 27 Jan 2023 00:00:00 +0000

This network performs 3D object detection using stereo vision, and belongs to the image-based result-lifting methods in 3D detection, which decompose 3D detection into two parts: 2D object detection and depth estimation.

Stereo RCNN extends Faster RCNN, as shown below:

Backbone

The authors use ResNet-101 combined with FPN, with shared weights. This backbone processes the left and right images at the same time, and the results are then merged as the input of Stereo RPN.

OTA

Thu, 06 Oct 2022 00:00:00 +0000

The label assignment problem in object detection

The PointNet Series

Sat, 07 Aug 2021 00:00:00 +0000

PointNet, PointNet++ and Frustum PointNet. The commonly used 3D point cloud object detection.

PointNet

PointNet applies deep learning directly to point cloud data. Point cloud data is a subset of the points in Euclidean space, and has three properties: disorder, the spatial relationship between points, and invariance.

Disorder. Point cloud data is a set and is insensitive to the order of the data, which requires the model to be invariant to different permutations of the data. There are several approaches. The first is to sort the point cloud by some rule, but for high-dimensional space it is very hard to find a stable sorting rule. Another is to treat it as a sequence and train it with an RNN, but this approach struggles to handle input elements of extremely large length (thousands to tens of thousands, as in a point cloud). Finally the authors use a symmetric function to aggregate the information of each point: $f(\lbrace x_1,x_2,\cdots,x_n\rbrace)\approx g(h(x_1),h(h(x_2),\cdots,h(x_n))$, where the left side of the equation is the goal and the right side is the designed symmetric function. From this we can see that each point in the point cloud is processed separately with $h$ and then passed into the symmetric function $g$. In the paper the authors choose $h$ to be an MLP and $g$ to be max pooling.
The spatial relationship between points. An object usually consists of point clouds in a specific space, and there are spatial relationships between points; PointNet exploits these relationships by concatenating global features with local features.
Invariance. The object represented by point cloud data should be invariant to some spatial transformations (rigid transformations). PointNet aligns the point cloud data before extracting features to guarantee invariance. The alignment is achieved by training a small network (T-Net) to obtain a transformation matrix and multiplying it with the input point cloud data. This operation can be seen as adjusting the spatial points; intuitively it rotates them into an orientation that is favorable for classification or segmentation (a rigid transformation). In the actual implementation a related loss function is also added to make the transformation matrix approximately orthogonal, so as to reduce the loss of point cloud information. (Regularization: $L_{reg}=\lVert I-AA^T\rVert_F^2$, that is, an L2 loss)

The rough structure of PointNet is as follows:

Multi-View 3D Object Detection Network

Fri, 06 Aug 2021 00:00:00 +0000

MV3D

SORT and DeepSORT

Thu, 05 Aug 2021 00:00:00 +0000

The multi-object tracking frameworks commonly used in industry at present.

SORT

This algorithm is based on an object detection network (Faster Rcnn) and makes use of the Kalman filter and the Hungarian algorithm (or the KM algorithm), greatly improving the speed of multi-object tracking. The rough pipeline is as follows:

The object detection algorithm obtains the object boxes Detections of the current frame, the Kalman filter obtains the Tracks of the current frame, and IOU matching is performed between the Detections and the Tracks. The final results fall into three categories:

Dynamic Head

Wed, 04 Aug 2021 00:00:00 +0000

Dynamic network head

Dynamic Convolution and Dynamic ReLU

Tue, 03 Aug 2021 00:00:00 +0000

Make it dynamic.

Dynamic Convolution

Ordinary static convolution applies the same kernel to all input images, whereas dynamic convolution adapts to different images and processes them with more suitable convolution parameters, which increases the expressive power of the model without increasing network depth or width.

Dynamic convolution dynamically aggregates multiple parallel convolution kernels according to attention. The attention dynamically adjusts the weight of each kernel according to the input, thereby producing an adaptive dynamic convolution. Because the attention is a function of the input, dynamic convolution is no longer a linear function. Superimposing kernels in a nonlinear way through attention gives stronger representational power. The rough structure is as follows:

CRF and Its Application in Segmentation

Sun, 16 May 2021 00:00:00 +0000

Conditional random field

Basic Networks for Image Segmentation

Fri, 30 Apr 2021 00:00:00 +0000

Image segmentation is pixel-level classification of an image, and mainly includes semantic segmentation and instance segmentation. Semantic segmentation only requires segmentation at the pixel level, while instance segmentation additionally requires separating different individuals on top of semantic segmentation. This post briefly introduces several basic semantic segmentation networks.

FCN

FCN (Full Convolution Network) is a classic image segmentation network whose main structure is an encoder-decoder, where the encoder extracts image feature information and the decoder interprets the extracted features. FCN replaces the fully connected layers in the network with convolution layers, so that 2D feature maps are obtained; these are then upsampled through deconvolution (transposed convolution), and finally a softmax layer produces the mask for each class. It can also handle input images of different sizes. The rough structure is shown below.

Object Detection

Fri, 20 Nov 2020 00:00:00 +0000

Object Detection in 20 Years

Anchor free--Centernet

Sun, 25 Oct 2020 00:00:00 +0000

Objects as Points

Some Common Loss Functions

Sun, 18 Oct 2020 00:00:00 +0000

some loss function

The Efficient Model Family

Sat, 03 Oct 2020 00:00:00 +0000

EfficientX basic family

Several Kinds of IOU Loss

Sat, 26 Sep 2020 00:00:00 +0000

IoU is an important metric in object detection; it measures how good a predicted box is by computing the ratio of the intersection to the union of the predicted box and the ground truth box. But the usual bounding box adjustment function generally adopts the L2 norm, that is, the loss is computed in an MSE manner, and some studies have shown that this is not the optimal optimization function, hence the IOU loss functions.

MobileNetV2

Sun, 26 Apr 2020 00:00:00 +0000

Inverted Residuals and Linear Bottlenecks

Mask RCNN

Mon, 10 Feb 2020 00:00:00 +0000

without bells and whistles

YOLO_v3

Fri, 17 Jan 2020 00:00:00 +0000

An Incremental Improvement

YOLO_v2 and YOLO9000

Fri, 10 Jan 2020 00:00:00 +0000

Better, Faster, Stronger

YOLO_v1

Wed, 01 Jan 2020 00:00:00 +0000

You Only Look Once

SSD

Thu, 03 Oct 2019 00:00:00 +0000

Single Shot MultiBox Detector

Faster RCNN

Sun, 22 Sep 2019 00:00:00 +0000

An object detection algorithm

Fast RCNN

Sat, 21 Sep 2019 00:00:00 +0000

A classic object detection algorithm

RCNN

Thu, 19 Sep 2019 00:00:00 +0000

A classic object detection algorithm

ResNet

Tue, 10 Sep 2019 00:00:00 +0000

First of all, happy Teachers’ Day to all teachers, and thank you for your spirit of giving roses to others with a lingering fragrance on your hands. Then, after almost a month, I have rolled back to update the blog again; before that I was busy with joining the company, hoping to fit into the company’s development work as soon as possible, so I was “busy” for a month. Next let us continue with the introduction of classic image recognition network models. In previous posts we already introduced the AlexNet and VGG networks, both of which pushed the depth of neural networks of that time to the extreme, and subsequent neural network models basically performed no better. This is because as the network gets deeper, during error backpropagation the weights of the earlier network layers basically do not get updated and stay around some fixed value. In 2015, Kaiming He proposed a shortcut connection that propagates the error to the earlier network layers, greatly improving network performance. Below is a brief analysis of the ResNet network.

Telling Apart Several Normalization Algorithms

Sun, 11 Aug 2019 00:00:00 +0000

Neural networks already have several normalization algorithms. They all look very similar and basically share a unified form. This post refers to the Zhihu column SIGAI’s article on normalization algorithms; the link is at the bottom of the post.

The basic problem

When using a neural network to learn from a dataset, because the sources of the data may differ considerably, the values of different attributes may also span very different ranges; all of this causes errors in how the data is treated and leads to poor learning by the network. To solve this problem we normalize the data, that is, we regularize data whose value ranges vary widely into the same scale, which is beneficial for the network’s learning. Up to now normalization has many variants, the common ones being Batch Normalization (BN), Layer Normalization (LN), Instance Normalization (IN) and Group Normalization (GN). In fact their processing is similar (subtract the mean, divide by the standard deviation, then apply a linear mapping); the difference lies in the dimension along which the statistics are computed. The concrete formula is as follows:

VGG

Tue, 30 Jul 2019 00:00:00 +0000

A classic convolutional neural network

AlexNet

Mon, 22 Jul 2019 00:00:00 +0000

A classic convolutional neural network

LightGBM Model

Sun, 24 Mar 2019 00:00:00 +0000

A Highly Efficient Gradient Boosting Decision Tree

Xgboost Model

Sun, 17 Mar 2019 00:00:00 +0000

A Scalabe Tree Boosting Systeam

Deep Learning Optimization

Sat, 03 Nov 2018 00:00:00 +0000

Optimization is a very difficult problem; usually one has to carefully design an objective function and related constraints, and also make sure the problem to be optimized is a convex function, but most real-world problems are non-convex. For non-convex optimization problems there are some basic optimization methods: SGD, Momentum, AdaGrad, RMSProp, Adam and so on

Stochastic Gradient Descent (SGD)

// stochastic gradient descent

Given the dataset $X=\\{x^{(1)},x^{(2)},...,x^{(n)}\\}$ and the dataset labels $Y=\\{y^{(1)}, y^{(2)},...,y^{(n)}\\}$

How the FFM Model Works

Sun, 28 Oct 2018 00:00:00 +0000

CTR (click-through rate) prediction plays an important role in content recommendation and delivery, and the FFM model is good at handling this kind of data. The FFM model is a variant of the FM model, or rather the FM model is a special case of the FFM model

Rough pipeline

Convert the dataset format (using the LIBSVM or FFMs style)
Compute the loss function (using the exponential loss function)
Optimize the parameters (using the AdaGrad + SG strategy)

Converting the data format

LIBSVM format (does not store zero-valued features):

Decision Tree

Wed, 29 Aug 2018 00:00:00 +0000

Decision tree, a common machine learning method, is somewhat like a flowchart. A sample is divided into a set according to the value it takes on some feature, and this is done recursively

Basic algorithm

Input: training set $D=\\{(\mathbf{x_1},y_1),(\mathbf{x_2},y_2),...,(\mathbf{x_m},y_m)\\}$; attribute set $A=\\{a_1,a_2,...,a_d\\}$

Output: a decision tree with node as the root node

Simple process:

Using some splitting criterion, the optimal feature is selected to split the samples, and this is done recursively, finally yielding the branch nodes of the decision tree

A Scikit-learn Cheat Sheet Collection

Sun, 26 Aug 2018 00:00:00 +0000

This time I am putting together the commonly used scikit-learn packages as a memo

K-nearest neighbors:

from sklearn.neighbors import KNeighborsClassifier

Linear regression:

from sklearn.linear_model import LinearRegression

Logistic regression:

from sklearn.linear_model import LogisticRgression

Decision tree:

from sklearn.tree import DecisionTreeClassifier

Naive Bayes:

Gaussian distribution: from sklearn.neive_bayes import GaussianNB
Multinomial distribution: from sklearn.neive_baybes import MultinomnalNB
Bernoulli distribution: from sklearn.neive_baybes import BernoulliNB

Support vector machine:

Classification: from sklearn.svm import SVC
Regression: from sklearn.svm import SVR

Ensemble algorithms (sklearn.ensemble):

bagging:

Classification: from sklearn.ensemble import BaggingClassifier

Bayesian Classification

Sun, 19 Aug 2018 00:00:00 +0000

Bayesian decision theory considers how to choose the optimal class label based on the relevant probabilities and misclassification losses

Suppose there are N possible class labels, that is $\mathbf{y}=\\{c_1,c_2,...,c_N\\}$, and $\lambda_{ij}$ is the loss incurred by misclassifying a sample whose true label is $c_j$ as $c_i$. Based on the posterior probability $P(c_i\vert\mathbf{x})$, we can obtain the expected loss incurred by classifying sample $\mathbf{x}$ as $c_i$, that is, the “conditional risk” on sample $\mathbf{x}$:

SVM Support Vector Machine

Thu, 16 Aug 2018 00:00:00 +0000

SVM is a very important model in machine learning, proposed by Vladimir Vapnik, the father of statistical learning theory

This post takes binary classification with SVM as the example

Derivation

Given the training dataset $D=\\{(\mathbf{x_1},y_1),(\mathbf{x_2},y_2),...,(\mathbf{x_m},y_m)\\}, y_i \in \\{-1, +1\\}$, the most basic idea of a classifier is to find in the sample space a dividing space based on the training set D that separates the samples of different classes. The dividing hyperplane can be described by the following linear equation:

Linear Regression and Logistic Regression

Mon, 13 Aug 2018 00:00:00 +0000

I have not updated the blog for a long time, having paused for about half a year, and I have also read half a year’s worth of books on machine learning and deep learning; now it is time to write some output

Linear regression

Given a dataset $D=\\{(\mathbf{x_1},y_1),(\mathbf{x_2},y_2),...,(\mathbf{x_m},y_m)\\}$, where $\mathbf{x_i}=(x_{i1};x_{i2};...;x_{id}), \ y_{i} \in R$. Linear regression tries to learn a linear model

$$f(x_i)=wx_i+b$$

such that $f(x_i)\simeq y_i$.

Derivation

Consider using MSE (mean squared error) to measure how good the model is; then the loss function $L(w,b)$ is:

Using g++ and gdb

Fri, 02 Mar 2018 00:00:00 +0000

G++ is the C++ compiler under Unix/Linux, and gdb is a powerful debugging tool under Unix/Linux. Most of the time, the C/C++ code we write does not run on Windows but on a Linux system. Here I record some usage of g++ and gdb in case they are needed.

Using g++

When g++ performs compilation work, it needs 4 steps in total:

Preprocessing, generating a .i file, mainly expanding header files and macro definitions: g++ -E test.cpp -o test.i
Compilation, converting the preprocessed file into assembly language and generating a .s file: g++ -S test.i -o test.s
Assembly, turning the assembly into object code (machine code) and generating a .o file, mainly converting the .s file into a binary file: g++ -c test.s -o test.o
Linking, linking the object code to generate an executable program: g++ test.o -o test

The steps above can also be done with a single command:

Configuring Django on Ubuntu

Mon, 19 Feb 2018 00:00:00 +0000

At the time of the 2018 Spring Festival, heaven knows what I went through, posting a blog article on the fourth day of the new year. Django was actually configured before the New Year; I had just learned some knowledge about developing web applications with python at that time, so I configured it along the way.

Environment used for the configuration

System: Ubuntu16.04 (LTS)
python version: python3.6 (using Anaconda to create a virtual environment)

Preparation

Uninstalling an old version of Django

If you are upgrading from an older version of Django, you need to deal with the old version a little.

Configuring Tensorflow on Linux

Sun, 04 Feb 2018 00:00:00 +0000

Borrowing a line from the TensorFlow Chinese community: TensorFlow, an open-source magic tool for artificial intelligence.

Environment used for the configuration

System: Ubuntu16.04 (LTS)
python version: python3.6 (using Anaconda to create a virtual environment)
Nvidia card: Geforce 920M (if there is no Nvidia card, only the CPU version can be installed)

Installing Tensorflow

tensorflow has two versions, CPU and GPU; I created two python virtual environments and installed these two versions separately. Below I record this painful installation process

Linux Penetration Testing Commands

Sat, 20 Jan 2018 00:00:00 +0000

Below is a penetration testing memo for Linux machines; I came across it on the internet by chance and copied it over here.

Penetration testing a Linux machine

Command	Description
netstat -tulpn	Show network ports in Linux together with the corresponding ID (PID)
watch ss -stplu	Observe TCP and UDP ports in real time through sockets
lsof -i	Show established connections
macchanger -m MACADDR INTR	Change the MAC address on KALI Linux
ifconfig eth0 192.168.2.1/24	Set the IP address in Linux
ifconfig eth0:192.168.2.3/24	Add an IP address to an existing network interface in Linux
ifconfig eth0 hw ether MAC ADDR	Change the address in Linux using ifconfig
ifconfig eth0 mtu 1500	Change the MTU size with ifconfig; change 1500 to the MTU you want
dig -x 192.168.1.1	Perform a reverse lookup on an IP address
host 192.168.1.1	Perform a reverse lookup on an IP address; useful when dig is not installed
dig @192.168.2.2 domain.com -t AXFR	Perform a DNS zone transfer with dig
host -l domain.com nameserver	Perform a DNS zone transfer with host
nbtstat -A x.x.x.x	Obtain the domain name corresponding to an IP address
ip addr add 192.168.2.22/24 dev eth0	Add a hidden IP address that does not show up when running ifconfig
tcpkill -9 host goole.com	Block access to goole.com from the host
echo "1">/proc/sys/net/ipv4/ip_forward	Enable IP forwarding, turning the Linux box into a router — which makes it convenient to control the traffic routed through this box
echo "8.8.8.8">/etc/resolv.conf	Use Goole's DNS

System information commands

Very helpful for local enumeration

MySQL Data File Paths on Linux

Fri, 01 Dec 2017 00:00:00 +0000

Server version: 5.7.20-0ubuntu0.16.04.1 (Ubuntu)

After logging into MySQL, you can use the following command to query where the related files are stored:

show variables like '%dir%';

The output is shown below:

Usually these are:

The default location of the database files is: /usr/share/mysql/ Configuration file: /etc/mysql/my.cnf

Database directory: /var/lib/mysql Configuration directory: /etc/share/mysql (mysql.server command and configuration files) MySQL related commands: /usr/bin Startup scripts: /etc/init.d/

One way to start mysql safely: /usr/bin/mysqld_safe-user=root&

Some Linux Commands

Tue, 07 Nov 2017 00:00:00 +0000

Some little-known Linux commands

Detecting Traffic Violations with Opencv

Sun, 05 Nov 2017 00:00:00 +0000

Opencv contains more than 2500 functions, and almost any mature algorithm you can think of can be implemented by calling Opencv’s functions, which is super convenient. This post combines Python and Opencv to detect cars in surveillance video and check whether they commit a violation (crossing the double yellow line).

System environment: Ubuntu16.04LTS+OpenCV3.3.0+Python2.7.12

Background extraction

For surveillance the position is relatively fixed, so the background will not change considerably over a long period of time. We can therefore consider using background subtraction to roughly extract the background.

Installing OpenCV on Linux

Fri, 20 Oct 2017 00:00:00 +0000

Recently, because a project needed computer vision recognition, I started tinkering with OpenCV. Although the installation process was long and winding, the experience of using it together with Python is actually quite good. Here I record the OpenCV installation and compilation process.

System environment: Ubuntu16.04

Software installed: OpenCV3.3.0+Python2.7.12

Updating the system packages:

$ sudo apt update
$ sudo apt upgrade

Installing some developer tools:

$ sudo apt-get install build-essential cmake pkg-config

The pkg-config package is very likely already installed by default, but just in case, add it in as well. The cmake program is used later to compile OpenCV.

Compiling C/C++ under Vscode

Sun, 01 Oct 2017 00:00:00 +0000

Visual Studio Code is a lightweight editor released by Microsoft; by installing various plugins it can be turned into a powerful multi-language, cross-platform compiler. Recently, because of practical needs, I had to debug some C/C++ code, so I started tinkering with Vscode. Below is a record of the problems that troubled me during the process.

The system used in this post is ubuntu16.04LTS

I will not go into the details of installing Vscode on linux, as Baidu has plenty of resources on that.

Python Classes

Sun, 20 Aug 2017 00:00:00 +0000

Life is short, I use Python

Python, Object Oriented

Thu, 10 Aug 2017 00:00:00 +0000

Life is short, I use Python

A Basic Git Tutorial

Tue, 04 Jul 2017 00:00:00 +0000

Some basic Git operations