DRE 7053 Generative Models

DRE 7053 Generative Models

Course code: 
DRE 7053
Department: 
Economics
Credits: 
5
Course coordinator: 
Rogelio Andrade Mancisidor
Course name in Norwegian: 
Generative Models
Product category: 
PhD
Portfolio: 
PhD Economics courses
Semester: 
2024 Spring
Active status: 
Active
Level of study: 
PhD
Teaching language: 
English
Course type: 
One semester
Introduction

Generative models are used in different fields of machine learning, e.g., image processing, natural language processing, representation learning, and multimodal learning just to name a few. Advances in parameterizing these models using deep neural networks have enabled scalable modeling of complex and high-dimensional data. This course focuses on Variational Autoencoders and Variational Diffusion models. The course consists of 5 days of teaching with both lectures and practical components, from June 10th.-14th at UiA.

Learning outcomes - Knowledge

The students are able to

  • Describe variational inference
  • Derive the evidence lower bound for different models
  • Understand the limitations of variational-based generative models
  • Understand the differences between variational autoencoders and diffusion models
Learning outcomes - Skills

The students are able to  

  • Apply generative models to new applications and problem settings
  • Implement generative models in Python libraries like TensorFlow
  • Explain the underlying theory in variational-based generative models
General Competence

Students will be able to describe the fundamentals of variational generative models and will be able to apply these methods in different contexts and applications.

Course content
  • Variational Inference
    • Mean field
    • Stochastic variational inference
    • Amortized inference
  • Variational Autoencoder (VAE)
    • Deriving and interpreting the evidence lower bound
    • Representation Learning
    • Semi-supervised VAE
    • Supervised VAE
    • Multimodal Learning
    • Well known problems: Posterior collapse, beta-annealing, upper-bound on mutual information
  • Diffusion Models
    • Diffusion principle
    • Variational Diffusion Models
Teaching and learning activities

This course is offered as an intensive course in a one-week period that combines reading, lectures, discussion sessions, and lab hours. In the lab sessions students will learn how to use Tensorflow to implement variational-based generative models. It is expected that some of the reading material will be read before starting the course.

 

Software tools
Phyton
Additional information

Good command of Python programming, ideally using libraries like TensorFlow or PyTorch.

Qualifications

Admission to a PhD Programme is a general requirement for participation in PhD courses at BI Norwegian Business School.

Strong knowledge on mathematical statistics and probability theory. Good command of Python programming, ideally using libraries like TensorFlow or PyTorch.

Assessments
Assessments
Exam category: 
Submission
Form of assessment: 
Submission PDF
Exam/hand-in semester: 
First Semester
Grouping: 
Individual
Duration: 
2 Month(s)
Comment: 
Home exam
Exam code: 
DRE 70531
Grading scale: 
Pass/fail
Resit: 
Examination when next scheduled course
Type of Assessment: 
Ordinary examination
Total weight: 
100
Student workload
ActivityDurationComment
Teaching
25 Hour(s)
Seminar groups
5 Hour(s)
Reading discussions
Student's own work with learning resources
40 Hour(s)
Group work / Assignments
10 Hour(s)
Lab sessions
Examination
50 Hour(s)
Project paper
Sum workload: 
130

A course of 1 ECTS credit corresponds to a workload of 26-30 hours. Therefore a course of 5 ECTS credit corresponds to a workload of at least 135 hours.

Semesters

Translations