Schedule

Unless noted otherwise, all readings are from Reinforcement Learning: An Introduction, 2nd Ed., Sutton and Barto

Date Topic/Notes Reading Assignment due
9/6 Introduction to RL SB 1.1--1.6 Self Assessment (Solutions).
9/10 Bandit Problems SB 2.1--2.10 Bandits quiz on Blackboard DUE; Bandits Assignment OUT
9/13 Bandit Problems

9/17 MDPs SB 3.1--3.8 MDPs quiz on Blackboard DUE; MDP Assignment OUT
9/20 MDPs
Bandits Assignment DUE
9/24 Dynamic Programming SB 4.1--4.8 Dynamic Programming Quiz on blackboard DUE
9/27 Dynamic Programming
Dynamic programming assignment OUT; MDP Assignment DUE
10/1 Monte Carlo SB 5.1--5.7 (you can skip Example 5.5) Monte Carlo Quiz on blackboard DUE
10/4 Off Policy Monte Carlo
Dynamic Programming assignment DUE; Monte Carlo assignment OUT
10/8 Temporal Difference Learning SB 6.1--6.8 Blackboard quiz on TD learning due
10/11 Review!
Monte Carlo assignment DUE; TD Learning assignment OUT
10/15 Midterm 1

10/18 Temporal Difference Learning

10/22 Temporal Difference Learning/Planning and Learning SB 8.1--8.6;8.9--8.12 Blackboard quiz on Planning and Learning due; Project Proposal DUE
10/25 Planning and Learning
TD Learning assignment DUE; Planning and Learning assignment OUT
10/29 Planning and Learning

11/1 Linear function approximation SB 9.1--9.5, 9.8 Linear function approximation Quiz due; Planning and Learning assignment DUE
11/5 Deep Learning Overview/ DQN GBC, 6.1--6.4, 9.1--9.3, Mnih, 2014 (DQN), DQN assignment OUT
11/8 DQN and extensions Hasselt, 2015 (Double DQN), Schaul, 2016 (Prioritized Replay), Wang, 2015 (Dueling) Mnih, 2016 (A3C)
11/12 Policy gradient and actor critic SB 13.1--13.7 Blackboard quiz on Policy Gradient due; DQN assignment DUE; Policy Gradient assignment OUT
11/15 Deep policy gradient and actor critic Silver, 2014 (DPG), Lillicrap, 2016 (DDPG), Mnih, 2016 (A3C)
11/19 Partially observable RL   Policy Gradient assignment DUE
11/22 Midterm 2

11/26 Multi-agent RL

11/29 THANKSGIVING BREAK

12/3 Project presentations

12/6 Project presentations

12/11 (No class)
Final Project DUE


Important note: all readings and assignments are due the day they appear on the schedule.