CSC412/2506 Winter 2023
Probabilistic Machine Learning
The language of probability allows us to coherently and automatically account for uncertainty. This course will teach you how to build, fit, and do inference in probabilistic models. These models let us generate novel images and text, find meaningful latent representations of data, take advantage of large unlabeled datasets, and even let us do analogical reasoning automatically. This course will teach the basic building blocks of these models and the computational tools needed to use them.
What you will learn
- Standard statistical learning algorithms, when to use them, and their limitations.
- The main elements of probabilistic models (distributions, expectations, latent variables, neural networks) and how to combine them.
- Standard computational tools (Monte Carlo, Stochastic optimization, regularization, automatic differentiation).
Instructor
- Michal Malyska,
- Email: michal.malyska@mail.utoronto.ca (put “CSC412” in the subject)
-
Office hours:
- Wednesdays 11:00AM - 12:00PM, location BA2283 and zoom
- Fridays 4PM-5PM, zoom
Reading
No required textbooks.
- (PRML) Christopher M. Bishop (2006) Pattern Recognition and Machine Learning
- (DL) Ian Goodfellow, Yoshua Bengio and Aaron Courville (2016), Deep Learning
- (MLPP) Kevin P. Murphy (2013), Machine Learning: A Probabilistic Perspective
- (PMLAT) Kevin P. Murphy (2023), Probabilistic Machine Learning: Advanced Topics
- (ESL) Trevor Hastie, Robert Tibshirani, Jerome Friedman (2009) The Elements of Statistical Learning
- (ISL) Gareth James, Daniela Witten, Trevor Hastie, Robert Tibshirani (2017) Introduction to Statistical Learning
- (ITIL) David MacKay (2003) Information Theory, Inference, and Learning Algorithms
Evaluation
- Assignment 1 (13.3%) - Released Jan 23rd, Due Feb 5th 11:59PM
- Assignment 2 (13.3%) - Released Feb 6th, Due Feb 19th 11:59PM
- Midterm (20%) - During the Monday and Wednesday lectures Feb 27th, Mar 1st
- Assignment 3 (13.4%) - Released Mar 9th, Due Mar 27th 11:59PM
- Final (40%) - April 24th 9:00AM
Assignments
Tentative Schedule
Week 1 - Jan 9th - Course Overview and Graphical Model Notation
Coverage
- Class Intro
- Topics covered
- Exponential Family
Materials
- Intro Slides
- Tutorial Materials
- Last year first lecture p1
- Last year first lecture p2
- First Lecture video from STA414
- Last year STA414 first lecture part 1
- Last year STA414 first lecture part 2
Week 2 - Graphical Model Notation, Decision Theory and Parametrizing Probabilistic Models
Week 3 - Latent variables and Exact Inference
Week 4 - Message Passing + Sampling
- Monday Slides
- Wednesday Slides
- Monday Recording
- Wednesday Recording
- Worked Example of message passing on tree
- Worked Example of Loopy BP
Week 5 - MCMC
Week 6 - Variational Inference
- Monday Slides
- Interactive HMM demo Note that it has a bunch of typos and errors.
- Wednesday Slides
- Friday Tutorial
Week 7 - Feb 21st & 22nd - No classes - Reading week
- Practice Midterm Note that this will be split into 2 x 4 question tests
- Practice Midterm Answers
Enjoy!
Week 8 - Midterm Week
Week 9 - Stochastic Variational Inference and Variational Autoencoders
- Monday Slides
- Monday Recording
- Reparametrization Trick
- Log Derivative Trick
- Wednesday Slides
- What is a VAE tutorial
- Wednesday Recording
- Tutorial
Week 10 - Embeddings
Week 11 - Kernel Methods, Attention
- Monday Slides
- Wednesday Slides
- Tutorial
- Self-practice (last year’s assignment) - not mandatory / not graded