Generative Adversarial Nets
Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Benigo, In NIPS, pp :2672-2680,2014.
[Python] [Hacks]
Generative Adversarial Nets(GAN) trains two models- generative model(G) that captures the data distribution and discriminative model(D) estimates whether the sample came from training data or from G. The training procedure for G is to maximize the probability of D making a mistake.It is based on minmax game rather than an optimization problem, and have a value function that one agent seeks to maximize and other seeks to minimize.In GAN's , one network produces a rich high dimensional vector that is used as the input to the other network, and attempts to choose an input that the other network does not know how to process.
D is trained to maximize the probability of assigning correct labels to both training examples and samples from G. And G is simultaneously trained to minimize log(1-D(G(Z))). When G is poor, D rejects samples with high confidence since it is different from the training data. In that case , G is trained to maximize log D(G(z)) rather than to minimize log(1-D(G(Z))). Generator nets used a mixture of ReLu and sigmoid activations, while discriminator used maxout activations and dropout was used during training.
GAN's are trained by simultaneously updating the discriminative distribution.The discriminator is updated by ascending its stochastic gradient and generator is updated by descending its stochastic gradient.
GAN's are primarily computational and can represent very sharp, degenerate distributions while other methods be somewhat blurry.
Terms:
Generative model(G): Captures the data distribution(trying to imitate and produce fake currency, using it without detection).
Discriminative model(D): Estimates the probability of the sample that came from the training data rather than G(trying to detect the fake currency).
Adversarial nets: Special case where the generative model generates sample by passing random noise through a multilayer perceptron.
Generative Adversarial Nets(GAN) trains two models- generative model(G) that captures the data distribution and discriminative model(D) estimates whether the sample came from training data or from G. The training procedure for G is to maximize the probability of D making a mistake.It is based on minmax game rather than an optimization problem, and have a value function that one agent seeks to maximize and other seeks to minimize.In GAN's , one network produces a rich high dimensional vector that is used as the input to the other network, and attempts to choose an input that the other network does not know how to process.
D is trained to maximize the probability of assigning correct labels to both training examples and samples from G. And G is simultaneously trained to minimize log(1-D(G(Z))). When G is poor, D rejects samples with high confidence since it is different from the training data. In that case , G is trained to maximize log D(G(z)) rather than to minimize log(1-D(G(Z))). Generator nets used a mixture of ReLu and sigmoid activations, while discriminator used maxout activations and dropout was used during training.
GAN's are trained by simultaneously updating the discriminative distribution.The discriminator is updated by ascending its stochastic gradient and generator is updated by descending its stochastic gradient.
GAN's are primarily computational and can represent very sharp, degenerate distributions while other methods be somewhat blurry.
Terms:
Generative model(G): Captures the data distribution(trying to imitate and produce fake currency, using it without detection).
Discriminative model(D): Estimates the probability of the sample that came from the training data rather than G(trying to detect the fake currency).
Adversarial nets: Special case where the generative model generates sample by passing random noise through a multilayer perceptron.