Deep Learning

For the classic diffusion model DDPM, during the reverse generation inference we need to restore random Gaussian noise $\mathbf{x}_T$ step by step into a meaningful image $\mathbf{x}_0$, requiring $T$ inference steps in total, so generation is slow. For this reason, DDIM (Denoising Diffusion Implicit Models) uses a smaller number of sampling steps to speed up inference, and the process of generating an image from random noise does not introduce additional noise, that is, the reverse process is deterministic. DDIM has the same optimization objective as DDPM, but does not require the forward process to be a Markov chain. ...

This is an image generation model of the diffusion model type. To put it simply, by continuously adding Gaussian noise to the original image $\mathbf{x}_0$ ($T$ rounds in total), $\mathbf{x}_0$ can finally be turned into pure Gaussian noise $\mathbf{x}_T$, and the model is responsible for restoring $\mathbf{x}_T$ to $\mathbf{x}_0$. This is illustrated below: Here the process from $\mathbf{x}_0$ to $\mathbf{x}_T$ is called the forward process, and the process from $\mathbf{x}_T$ to $\mathbf{x}_0$ is the reverse or inference process. ...

Previous diffusion models (DM) generally operate over the full image, which makes the GPU memory required to run a DM model very large. The latent diffusion model (LDM), by contrast, maps the input to a latent space through a pretrained autoencoder encoder, performs the DM operations in that latent space, and finally maps the output back to the original image space through the autoencoder decoder. The authors also introduce the conditioning information into the DM through cross attention, thereby achieving controllable image generation. ...

Ref: https://lilianweng.github.io/posts/2017-08-20-gan/ Some background knowledge For two distributions $P_1$ and $P_2$, their KL divergence (Kullback-Leibler divergence) and JS divergence (Jensen-Shannon divergence) can be used to describe the difference between the two distributions, specifically as follows: $$ \begin{aligned} D_{KL}(P_1 \vert\vert P_2) &= \mathbb{E}_{x\sim P_1}\left[\log\frac{P_1}{P_2}\right] = \int_x P_1\log\frac{P_1}{P_2}dx \\ D_{JS}(P_1 \vert\vert P_2) &= \frac{1}{2}D_{KL}\left(P_1\vert\vert\frac{P_1+P_2}{2}\right) + \frac{1}{2}D_{KL}\left(P_2\vert\vert\frac{P_1+P_2}{P_2}\right) \end{aligned} $$A GAN usually contains two models: a discriminator $D$ and a generator $G$. The output of the discriminator is the probability that a sample comes from the real data, that is, its purpose is to tell real data apart from generated data, while the purpose of the generator is to make the generated data as close as possible to the real data distribution, that is, to make it hard for the discriminator to tell them apart. ...