Schedule

Unless noted otherwise, all readings are from Reinforcement Learning: An Introduction, 2nd Ed., Sutton and Barto

Date Topic/Notes Reading
Assignment due
1/8 Introduction to RL SB 1.1--1.6
Self Assessment (Solutions)
1/12 Bandit Problems SB 2.1--2.10
 
1/15 Bandit Problems
0. Intro assignment DUE (Friday)
1/19 No class (MLK Day)  
 
1/22 MDPs SB 3.1--3.8
1. Bandits assignment DUE (Friday)
1/26 MDPs  
 
1/29 Dynamic Programming SB 4.1--4.8
2. MDP assignment DUE (Friday)
2/2 Dynamic Programming
 
2/5 Monte Carlo Methods SB 5.1--5.7 (you can skip Example 5.5)
3. DP assignment DUE (Friday)
2/9 Monte Carlo Methods   
Project description OUT
2/12 Temporal Difference Learning SB 6.1--6.8
4. MC assignment DUE (Friday)
2/16 No class (Presidents' Day) 


2/19 Temporal Difference Learning  
Project proposal DUE
2/23 Linear Function Approximation SB 9.1--9.5, 9.8, 10.1
5. TD Learning assignment DUE (Monday)
2/26 Exam 1 SB 8.1--8.6;8.9--8.12
3/2 No class (Spring break)

3/5

 No class (Spring break)

3/9 Deep Learning Overview GBC, 6.1--6.4, 9.1--9.3, (optional background)
3/12 Deep Q-learning (DQN) Mnih, 2014 DQN
7. Function approx. assignment DUE (Friday)
3/16 DQN and extensions Hasselt, 2015 (Double DQN), Schaul, 2016 (Prioritized Replay), Wang, 2015 (Dueling) Mnih, 2016 (A3C), Rainbow
3/19 Policy gradient and actor critic SB 13.1--13.7
8. DQN assignment DUE (Friday)
3/23 Deep policy gradient and actor critic Silver, 2014 (DPG), Lillicrap, 2016 (DDPG), Mnih, 2016 (A3C)
  
3/26 Planning and Learning SB 8.1--8.6;8.9--8.12  9. PG assignment DUE (Friday)
3/30 Planning and Learning  
4/2 Advanced Topics
6. Planning and Learning assignment DUE (Friday)
4/6 Exam 2
 
4/9
 Advanced Topics


4/13 Project Presentations


4/16
 Project Presentations


4/20 No class (Patriots' Day)


   


4/21
 Project Reports Due

Report due at 11:59 PM -- This is a hard deadline, no extensions


Important note: unless noted otherwise, all readings and assignments are due on the day that they appear in the schedule.