Schedule

Unless noted otherwise, all readings are from Reinforcement Learning: An Introduction, 2nd Ed., Sutton and Barto

Date Topic/Notes Reading
Assignment due
9/8 Introduction to RL SB 1.1--1.6
Self Assessment (Solutions)
9/12 Bandit Problems SB 2.1--2.10
Intro assignment DUE (Wednesday)
9/15 Bandit Problems
 
9/19 MDPs SB 3.1--3.8
Bandits assignment DUE (Wednesday)
9/22 MDPs

 
9/26 Dynamic Programming SB 4.1--4.8
MDP assignment DUE (Wednesday)
9/29 Dynamic Programming

 
10/3 Monte Carlo Methods SB 5.1--5.7 (you can skip Example 5.5)
DP assignment DUE (Wednesday)
10/6 Monte Carlo Methods
Project description OUT 
10/10 No class (Columbus/Indigenous People's Day) 

MC assignment DUE (Wednesday)
10/13 Temporal Difference Learning SB 6.1--6.8
 
10/17 Temporal Difference Learning

 
10/20 Exam 1  
Project proposal DUE
10/24 Planning and Learning SB 8.1--8.6;8.9--8.12
TD Learning assignment DUE (Wednesday)
10/27 Planning and Learning   

10/31 Planning and Learning 
 
11/3 Linear Function Approximation SB 9.1--9.5, 9.8, 10.1
 Planning and Learning assignment DUE
11/7 Linear Function Approximation

11/10 Deep Learning Overview/ DQN GBC, 6.1--6.4, 9.1--9.3, Mnih, 2014 (DQN)

11/14 DQN and extensions Hasselt, 2015 (Double DQN), Schaul, 2016 (Prioritized Replay), Wang, 2015 (Dueling) Mnih, 2016 (A3C)
 Function assignment DUE 
11/17 Policy gradient and actor critic SB 13.1--13.7
  
11/21 Deep policy gradient and actor critic Silver, 2014 (DPG), Lillicrap, 2016 (DDPG), Mnih, 2016 (A3C)
 DQN assignment DUE
11/24 No class (Thanksgiving)
 
11/28 Exam 2
PG assignment DUE
12/1
 Project Presentations


12/5
 Project Presentations


12/11
 Project Reports Due

Report due at 11:59 PM -- This is a hard deadline, no extensions


Important note: unless noted otherwise, all readings and assignments are due on the day that they appear in the schedule.