GAN

Ref: https://lilianweng.github.io/posts/2017-08-20-gan/

Some background knowledge

For two distributions $P_1$ and $P_2$, their KL divergence (Kullback-Leibler divergence) and JS divergence (Jensen-Shannon divergence) can be used to describe the difference between the two distributions, specifically as follows:

$$ \begin{aligned} D_{KL}(P_1 \vert\vert P_2) &= \mathbb{E}_{x\sim P_1}\left[\log\frac{P_1}{P_2}\right] = \int_x P_1\log\frac{P_1}{P_2}dx \\ D_{JS}(P_1 \vert\vert P_2) &= \frac{1}{2}D_{KL}\left(P_1\vert\vert\frac{P_1+P_2}{2}\right) + \frac{1}{2}D_{KL}\left(P_2\vert\vert\frac{P_1+P_2}{P_2}\right) \end{aligned} $$

A GAN usually contains two models: a discriminator $D$ and a generator $G$. The output of the discriminator is the probability that a sample comes from the real data, that is, its purpose is to tell real data apart from generated data, while the purpose of the generator is to make the generated data as close as possible to the real data distribution, that is, to make it hard for the discriminator to tell them apart.

The process above is realized by training $D$ and $G$ alternately. In practice the discriminator is usually trained first, then the generator, and this is repeated until the two models reach an equilibrium (Nash equilibrium).

Before getting to the main text, let me define some variables:

• $p_z$: the data distribution of the noise input $z$, usually a Gaussian distribution
• $p_g$: the data distribution generated by the generator
• $p_r$: the real data distribution

Original GAN

In the original GAN, for $D$ we want the discriminator’s score on real data to be as high as possible, that is, to maximize $\mathbb{E}_{x\sim p_r(x)}[\log D(x)]$, while for fake data we want the discriminator’s score to be as small as possible (close to 0), that is, to maximize $\mathbb{E}_{z\sim p_z(z)}[\log(1-D(G(z)))]$. For $G$, we want the generated data to be as close to the real data as possible (close to 1), which is reflected as minimizing $\mathbb{E}_{z\sim p_z(z)}[\log(1-D(G(z)))]$.

In summary, the loss function of the original GAN is:

$$ \begin{aligned} &\min_G \max_D L(D,G) \\ & \begin{aligned} L(D,G) &= \mathbb{E}_{x\sim p_r(x)}[\log D(x)] + \mathbb{E}_{z\sim p_z(z)}[\log(1-D(G(z)))] \\ &= \mathbb{E}_{x\sim p_r(x)}[\log D(x)] + \mathbb{E}_{x\sim p_g(x)}[\log(1-D(x))] \end{aligned} \end{aligned} $$

Taking the derivative of $L(D,G)$ with respect to $D$ gives the optimal discriminator $D$:

$$ D^\star(x) = \frac{p_r(x)}{p_r(x)+p_g(x)} \in [0, 1] $$

When the generator is optimal, $p_r = p_g$, so the discriminator is $\frac{1}{2}$.

When both the discriminator and the generator are optimal, that is, $D^\star(x)=\frac{1}{2}$, substituting into $L(G,D)$ gives the value of the loss function at the global optimum:

$$ \begin{aligned} L(G^\star,D^\star) &= \mathbb{E}_{x\sim p_r(x)}[\log D(x)] + \mathbb{E}_{x\sim p_g(x)}[\log(1-D(x))] \\ &= \int_x \bigg(p_r(x)\log(D^\star(x)) + p_g(x)\log(1-D^\star(x))\bigg)dx \\ &= \log\frac{1}{2}\int_x p_r(x)dx + \log\frac{1}{2}\int_x p_g(x)dx \\ &= -2\log2 \end{aligned} $$

Assuming the discriminator is optimal, rewriting the JS divergence gives:

$$ \begin{aligned} D_{JS}(p_{r} \| p_g) =& \frac{1}{2} D_{KL}(p_{r} || \frac{p_{r} + p_g}{2}) + \frac{1}{2} D_{KL}(p_{g} || \frac{p_{r} + p_g}{2}) \\ =& \frac{1}{2} \bigg( \log2 + \int_x p_{r}(x) \log \frac{p_{r}(x)}{p_{r} + p_g(x)} dx \bigg) + \\& \frac{1}{2} \bigg( \log2 + \int_x p_g(x) \log \frac{p_g(x)}{p_{r} + p_g(x)} dx \bigg) \\ =& \frac{1}{2} \bigg( \log4 + L(G, D^*) \bigg) \end{aligned} $$

that is, $L(G,D^\star) = 2D_{JS}(p_r\vert\vert p_g) - 2\log2$.

When the discriminator can accurately separate real data from fake data, we have $\forall x \in p_r, D(x) = 1$ and $\forall x \in p_g, D(x) = 0$, and substituting into $L(G,D)$ gives 0, that is, the gradient is 0.

From this we get the main reason why the original GAN is hard to train: when the discriminator converges very well, the generator’s gradient vanishes and the loss cannot decrease; when the discriminator does not converge well, the generator’s gradient provides no guidance. Only when the discriminator converges to a certain degree is the generator’s gradient effective and instructive, and that degree is very hard to control.

Later the authors proposed an improved loss function for the generator: $\mathbb{E}_{x\sim p_g}[-\log D(x)]$.

Rewriting the KL divergence gives:

$$ \begin{aligned} D_{KL}(p_g\vert\vert p_r) &= \mathbb{E}_{x\sim p_g}\bigg[\log\frac{p_g}{p_r}\bigg] \\ &= \mathbb{E}_{x\sim p_g}\bigg[\log\frac{p_g/(p_r+p_g)}{p_r/(p_r+p_g)}\bigg] \\ &= \mathbb{E}_{x\sim p_g}\bigg[\log\frac{1-D^\star(x)}{D^\star(x)}\bigg] \\ &= \mathbb{E}_{x\sim p_g}\log[1-D^\star(x)] - \mathbb{E}_{x\sim p_g}[\log D^\star(x)] \end{aligned} $$

Combining with $L(G, D^\star)$ above gives:

$$ \begin{aligned} \mathbb{E}_{x\sim p_g}[-\log D(x)] &= D_{KL}(p_g\vert\vert p_r) - \mathbb{E}_{x\sim p_g}[\log(1-D^\star(x))] \\ &= D_{KL}(p_g\vert\vert p_r) - 2D_{JS}(p_r\vert\vert p_g) + 2\log2 + \mathbb{E}_{x\sim p_r}[\log D^\star(x)] \end{aligned} $$

Since the last two terms do not depend on the generator, minimizing $\mathbb{E}_{x\sim p_g}[-\log D(x)]$ is ultimately equivalent to minimizing $D_{KL}(p_g\vert\vert p_r) - 2D_{JS}(p_r\vert\vert p_g)$.

This expression has to minimize the KL divergence between the generated distribution and the real distribution while at the same time maximizing the JS divergence between them, which is self-contradictory. Moreover, the JS divergence term makes model training unstable.

For the KL divergence term $D_{KL}(p_g\vert\vert p_r)$, we have:

• When $p_g(x) \rightarrow 0$ and $p_r(x)\rightarrow 1$, $p_g(x)\log\frac{p_g(x)}{p_r(x)}\rightarrow 0$, that is, the contribution to $D_{KL}(p_g\vert\vert p_r)$ tends to zero
• When $p_g(x) \rightarrow 1$ and $p_r(x)\rightarrow 0$, $p_g(x)\log\frac{p_g(x)}{p_r(x)}\rightarrow +\infty$, that is, the contribution to $D_{KL}(p_g\vert\vert p_r)$ tends to positive infinity

So the model penalizes the two errors above differently. The first error corresponds to the generator failing to generate real samples, and the penalty is tiny. The second error corresponds to the generator generating unrealistic samples, and the penalty is huge. With this behavior the model tends to generate some repetitive samples that are close to the real distribution, which makes the generated data lack diversity, that is, the mode collapse problem.

WGAN

Wasserstein GAN: https://zhuanlan.zhihu.com/p/25071913

Support: the subset of the non-zero part of a function; the support of a probability distribution is the set of all parts where the probability density is non-zero.

Manifold: a generalization of the concepts of curve and surface in high-dimensional space; we can understand this concept intuitively in lower dimensions. For example, we say a surface in 3D space is a 2D manifold, because its intrinsic dimension is only 2, and a point moving on this 2D manifold has only two degrees of freedom. Likewise, a curve in 3D or 2D space is a 1D manifold.

Measure: a generalization of the concepts of length, area and volume in high-dimensional space, which can be understood as hypervolume.

WGAN proposes some solutions to the problem that GANs are hard to train, the most important of which is replacing the KL and JS divergences with the Wasserstein distance.

Compared with the original GAN, the following changes are made:

• Remove the sigmoid from the last layer of the discriminator
• Do not take the log in the losses of the generator and discriminator
• After each update of the discriminator parameters, clip their absolute values to no more than a fixed constant
• Do not use momentum-based optimization algorithms (such as momentum and Adam); SGD, RMSProp and the like are recommended

But in practice WGAN still has training difficulties, mainly because it forcibly enforces the Lipschitz constraint on the discriminator’s gradient through weight clipping. So WGAN-GP was proposed afterwards.