DRE 7053 Generative Models
DRE 7053 Generative Models
Generative models are used in different fields of machine learning, e.g., image processing, natural language processing, representation learning, and multimodal learning just to name a few. Advances in parameterizing these models using deep neural networks have enabled scalable modeling of complex and high-dimensional data. This course focuses on Variational Autoencoders and Variational Diffusion models. The course consists of 5 days of teaching with both lectures and practical components.
The students are able to
- Describe variational inference
- Derive the evidence lower bound for different models
- Understand the limitations of variational-based generative models
- Understand the differences between variational autoencoders and diffusion models
The students are able to
- Apply different types of generative models for new applications, or problem settings, in a broad range of domains such as financial economics, credit risk, marketing analytics, image analysis, health analytics, multimodal settings, etc.
- Implement generative models in Python libraries like TensorFlow
- Explain the underlying theory in variational-based generative models
Students will be able to describe the fundamentals of variational generative models and will be able to apply these methods in different contexts and applications. Describe how VAE and Diffusion models use variational inference to derive their objective function.
- Variational Inference
- Latent Variable Models
- Variational Inference
- Mean field
- Stochastic variational inference
- Amortized inference
- Variational Autoencoder (VAE)
- Deriving and interpreting the evidence lower bound
- The representation trick
- Representation Learning
- Semi-supervised VAE
- Supervised VAE
- Multimodal Learning
- Well known problems: Posterior collapse, beta-annealing, upper-bound on mutual information
- Diffusion Models
- Denoising Diffusion Models
- Forward and Backward Diffusion Process
- Deriving the ELBO
This course is offered as an intensive course in a one-week period that combines reading, lectures, discussion sessions, and lab hours. In the lab sessions students will learn how to use Tensorflow to implement variational-based generative models. It is expected that some of the reading material will be read before starting the course.
Link to course material: https://github.com/BI-DS/DRE7053
Admission to a PhD Programme is a general requirement for participation in PhD courses at BI Norwegian Business School.
Strong knowledge on mathematical statistics and probability theory. Good command of Python programming, ideally using libraries like TensorFlow or PyTorch.
Strong knowledge of mathematical statistics and probability theory. Good knowledge of Python programming, ideally using libraries such as TensorFlow or PyTorch. Note, students who do not meet these requirements will be offered a 6-hour preparatory course in mathematical statistics and probability theory.
| Assessments |
|---|
Exam category: Submission Form of assessment: Submission PDF Exam/hand-in semester: First Semester Grouping: Individual Duration: 2 Month(s) Comment: Home exam Exam code: DRE 70531 Grading scale: Pass/fail Resit: Examination when next scheduled course |
| Activity | Duration | Comment |
|---|---|---|
Teaching | 35 Hour(s) | Teaching, lab sessions, and reading discussions. |
Student's own work with learning resources | 40 Hour(s) | |
Examination | 80 Hour(s) | Project paper |
A course of 1 ECTS credit corresponds to a workload of 26-30 hours. Therefore a course of 5 ECTS credit corresponds to a workload of at least 135 hours.