Generative Models: GANs, VAEs and Diffusion
How image generators work — and how to train small ones.
Taught by Rohan Maharjan · ML Engineer
What you'll learn
- Explain the likelihood, adversarial and score-based views of generative modelling and how they relate
- Implement and train a variational autoencoder in PyTorch and sample from its latent space
- Build a DCGAN, diagnose mode collapse and apply the stabilisation tricks that make it converge
- Implement a denoising diffusion model with a noise schedule, training loop and sampler
- Describe conditioning, classifier-free guidance and latent diffusion as used in text-to-image systems
- Evaluate generated samples with FID, precision and recall, and a structured human review
About this course
This generative models course is for machine learning engineers and advanced students who want to understand how modern image generators such as Stable Diffusion actually work, and to train small versions themselves. You will implement a variational autoencoder, a generative adversarial network and a denoising diffusion model in PyTorch, each on datasets small enough to train on a single GPU or a patient CPU. Along the way you will learn the maths that motivates each family, the training tricks that make GANs converge, how classifier-free guidance and latent diffusion turn a noise predictor into a text-to-image system, and how to evaluate generated samples with FID and human judgement. You should be comfortable with PyTorch, know what a convolution and a loss function are, and have trained a neural network before. By the end you will be able to read a generative modelling paper and reproduce its core idea in code.
Course content
7 lessons- 1What Generative Models LearnThe generative modelling problem, the three main families, and the shared tooling you will use throughout.Preview
- 2Autoencoders and VAEsFrom compression to generation: the reparameterisation trick, the ELBO and a working VAE in PyTorch.
- 3GANs: The Adversarial GameGenerator versus discriminator, the minimax objective, and a DCGAN that produces sharp 32-pixel images.
- 4Training GANs That ConvergeDiagnose mode collapse and instability, then apply spectral normalisation, hinge loss, R1 and other fixes.
- 5Diffusion Models: Noise In, Image OutThe forward noising process, the noise-prediction objective, and a compact DDPM training loop and sampler.
- 6Conditioning, Guidance and Latent DiffusionHow class and text conditioning, classifier-free guidance and latent-space diffusion produce text-to-image systems.
- 7Evaluating Generative ModelsFID, precision and recall, and structured human review, plus the pitfalls of each.