Announcements

  • Prerequisites: CS 5800 or CS 7800 with a minimum grade of C-. I will not enforce these pre-requisites this year. However, note that you are taking the course at your own risk. I will assume the knowledge of algorithms as well as intellectual capacity and work ethic of a student who passed such a course.  I cannot add you to the course directly. Please show this note to a person who can enroll you in the course if the system blocks you from doing it due to prerequisites. 
  • How to prepare for the course? See here for some guidance.
  • First class: Tuesday, January 19
  • Midterm exam: Week 8, Tuesday, in class.
  • Final exam: Friday, April 23, in class
  • Mini project report due: Monday, April 26
  • Final week office hours: Friday, April 30, 3:25-6:25pm ☀️
  • This class will be held Online.

———————————————————————

Last updated: April 20, 2021

Weekly Schedule

———————————————————————

Week 14, April 19 ☀️

Final exam, Friday, in class

Topics

  • Kernel machines for graphs
  • Review of topics for final exam

Reading materials

  • Vishwanathan et al. Graph kernels, J Mach Learn Res, 2010. See here.

Handouts and code

  • Kernel machines slides 

Mini project report instructions

  • Instructions for submission of the mini project report here ☀️

———————————————————————

Week 13, April 12

Topics

  • Support vector machines

Reading materials

  • Textbook #1 (Bishop): Sparse kernel machines (Chapter 7; 7.1)

Handouts and code

  • Support vector machines slides (last updated: 04/14/2021)

———————————————————————

Week 12, April 5

Topics

  • Committee machines
  • Principal Component Analysis (PCA)

Reading materials

  • Textbook #1 (Bishop): Combining models (Chapter 14)

Handouts and code

  • Committee machines slides 
  • Committee machines code 
  • Principal component analysis slides 

———————————————————————

Week 11, March 29

Topics

  • Classification and regression trees

Reading materials

  • Tan et al., Introduction to Data Mining (Chapter 4)
  • Mitchell: Machine learning (Chapter 3)

Handouts and code

  • Classification and regression trees slides 

———————————————————————

Week 10, March 22

Topics

  • Empirical evaluation

Reading materials

Handouts and code

  • Empirical evaluation slides 

Homework assignments

  • Assignment #4 available here.

———————————————————————

Week 9, March 15

Topics

  • Neural networks
  • Convolutional and deep neural networks

Reading materials

  • Textbook #1 (Bishop): Neural networks (Chapter 5)
    • Sections 5.1-5.3 (skip 5.3.4), 5.5 (skip 5.5.4 and on)
  • Lecture notes (Radivojac & White): representations 
  • The RPROP algorithm can be found here 
  • A nice online tutorial on CNNs here 

Handouts and code

  • Neural networks slides (last updated: 03/16/2021)
  • Convolutional and deep neural networks slides

———————————————————————

Week 8, March 8

Midterm Exam, Tuesday, in class

Topics

  • Data and data preprocessing

Handouts and code

———————————————————————

Week 7, March 1

Topics

  • K-means advanced concepts
  • Data and data preprocessing

Reading materials

Handouts and code

———————————————————————

Week 6, February 22

Topics

  • Logistic regression
  • Generalized linear models

Reading materials

  • Textbook #1 (Bishop): Linear models for classification (Chapter 4)
    • Sections 4.1 (4.1.1, 4.1.2, 4.1.3, 4.1.7), 4.3 (4.3.2, 4.3.3)
  • Lecture notes (Radivojac & White): generalized linear models

Handouts and code

  • Logistic regression slides 
  • Generalized linear models slides
  • Logistic regression code 

———————————————————————

Week 5, February 15

Topics

  • Basic principles of optimization
  • Perceptron

Reading materials

Handouts and code

Homework assignments

  • Assignment #2 available here.
  • Assignment #3 available here.

———————————————————————

Week 4, February 8

Topics

  • Linear regression
  • Mini-project discussion

Reading materials

  • Textbook #1 (Bishop): Linear models for regression (Chapter 3)
    • Sections 3.1, 3.2, 3.3 (light reading)
  • Lecture notes (Radivojac & White): linear regression 
  • Lecture notes (Radivojac & White: radial basis function networks

Handouts and code

  • Linear regression slides.
  • Linear regression for nonlinear problems slides. (last updated: 02/15/2021)
  • Mini-project slides.
  • Linear regression code.

———————————————————————

Week 3, February 1

Topics

  • Prediction
  • Naive Bayes models

Reading materials

  • Lecture notes (Radivojac & White): prediction (last updated: 02/02/2021)

Handouts and code

  • Prediction slides. (last updated: 02/07/2021)

———————————————————————

Week 2, January 25

Topics

  • Basics of parameter estimation
  • Expectation-maximization algorithm

Reading materials

  • Lecture notes (Radivojac & White): estimation 
  • Textbook #1 (Bishop): Mixture Models and EM (Chapter 9)
    • Sections 9.1, 9.2, 9.3, 9.4 (light reading)

Handouts and code

  • Parameter estimation slides.
  • EM algorithm code.

Homework assignments

  • Assignment #1 available here.

———————————————————————

Week 1, January 18

Topics

  • Class overview and logistics
  • Short review of probability theory

Reading materials

  • Textbook #1 (Bishop): Introduction (Chapter 1)
  • Lecture notes (Radivojac & White): probability

Handouts and code

  • Class overview slides.
  • Probability theory slides. (last updated: 01/23/2021)
  • Random variables slides. (last updated: 01/23/2021)

———————————————————————

———————————————————————