DS 4400: Machine Learning and Data Mining I

Spring 2021

Class Information

Calendar

Additional Reading

Other Resources

 

Instructors:

Class Schedule:

  • Tuesday 11:45am-1:25pm; Thursday 2:50-4:30pm EST
  • Location: Shillman Hall 320 (See Canvas for Zoom link)

Office Hours:

  • Alina: Tuesday 4:30-5:30pm EST; Thursday 4:45-5:45pm
  • Omkar: Monday and Wednesday, 3:00-4:00pm EST
  • Prabal: Monday and Thursday, 12:00-1:00pm EST
  • Saurabh: Friday, 10am-12pm EST
  • All meetings will be remote on Zoom, see Canvas for links

Class forum:  Piazza (See Canvas for link)

Class policies:  Academic integrity policy is strictly enforced

Class description:

Machine learning is a fast-pacing and exciting field achieving human-level performance in tasks such as image classification, speech recognition. machine translation, precision medicine, and self-driving cars. Machine learning has already impacted greatly our daily lives and has the potential to transform the world even more in the near future. This course will provide a broad introduction to machine learning and cover the fundamental algorithms for supervised learning. We will cover topics related to regression, linear classification, non-linear classification, ensemble models, and deep learning. The class will also provide an introduction into ethics and fairness concerns of machine learning, as well as adversarial machine learning, an emerging area that studies the fundamental security issues of machine learning.   

 

Pre-requisites:

 

  • Probability
  • Statistics
  • Linear algebra

 

Textbook

[ISL] Gareth JamesDaniela WittenTrevor Hastie and Robert Tibshirani. An Introduction to Statistical Learning with Applications in R.

Grading

The grade will be based on:

 

-       Assignments – 25%

-       Final project report and presentation – 30%

-       Midterm Exam – 20%

-       Final Exam – 20%

-       Class participation – 5%

 

 Calendar (Tentative)

 

Unit

Week

Date

Topic

Readings

1

Tue

01/19

Course outline (syllabus, grading, policies)

[PDF]

[ISL] Chapters 1 and 2.1

Introduction and Review

 

Thu

01/21

Classification and regression

Bias-variance tradeoff

[Lecture] [Annotations]

[ISL] Chapters 2.2.1 and 2.2.2

Probability review from Stanford

2

Tue

01/26

Probability and linear algebra review.

[Lecture] [Annotations]

Linear algebra review from Stanford

Thu

01/28

Simple linear regression

Closed from solution. Correlation

[Lecture] [Annotations]

[ISL] Chapter 3.1

Linear regression

3

Tue

02/02

Multiple linear regression

Closed form solution

[Lecture] [Annotations]

[ISL] Chapter 3.2

Thu

02/04

Gradient descent

[Lecture] [Annotations]

Lecture notes from Stanford on linear regression, part 1.1

HW 1 due on Feb 5

Regularization and cross-validation

4

Tue

02/09

Regularization.

Lasso and ridge regression

[Lecture] [Annotations]

[ISL] Chapter 6.2

Thu

02/11

k-Nearest Neighbors (kNN).

Cross-validation

Linear classification. Logistic regression

[Lecture] [Annotations]

[ISL] Chapter 5.1

[ISL] Chapter 4.1, 4.2, and 4.3

 

 

Linear   Classification

5

Tue

02/16

Logistic regression

Evaluation of ML, metrics

[Lecture] [Annotations]

Lecture notes from Stanford on linear regression, part 2

 

 

Thu

02/18

Project discussion

Evaluation of ML

ROC curves

[Lecture] [Annotations]

 

HW 2 due on Feb 19

Generative Models

6

Tue

02/23

Generative models

LDA

[Lecture] [Annotations]

[ISL] Chapter 4.4.1 and 4.4.2 for LDA

 

Thu

02/25

Naïve Bayes

Review

[Lecture] [Annotations]

7

Tue

03/02

Midterm exam

Tree and Ensemble Classification

 

Thu

03/04

Decision trees

Information Gain

[Lecture] [Annotations]

Chapter 8.1.2

Project proposal due: March 4

8

Tue

03/09

Ensemble learning

Bagging and boosting

[Lecture] [Annotations]

Chapter 8.2

HW 3 due: March 8

 

Thu

03/11

SVM

Linear SVM

[Lecture] [Annotations]

Chapter 9.1, 9.2

 

9

Tue

03/16

Kernel SVM

[Lecture] [Annotations]

Chapter 9.3, 9.4, 9.5

Deep learning

 

 

Thu

03/18

Neural networks and deep learning.

Feed-Forward Networks.

[Lecture] [Annotations]

 

10

Tue

03/23

Feed-Forward Networks.

Keras tutorial

[Lecture] [Annotations]

Stanford notes on deep learning, parts 1 and 2

Optional: Chapter 4 from Dive into Deep Learning

Thu

03/25

Convolutional Neural Networks

[Lecture] [Annotations]

Optional: Chapter 6 from Dive into Deep Learning

HW 4 due: March 26

SVM

11

Tue

03/30

Convolutional Neural Networks

Regularization in Neural Networks

[Lecture] [Annotations]

Project milestone due: March 31

 

Thu

04/01

Review for final exam

Transfer learning

[Lecture] [Annotations]

Ethics of ML and

12

Tue

04/06

Final exam

Adversarial ML

Thu

04/08

Ethics of ML/AI

 

13

Tue

04/13

Backpropagation

[Lecture] [Annotations]

 

 

Thu

04/15

Adversarial machine learning

[Lecture]

 

 

14

Tue

04/20

Project presentations, 11:45am -2:30pm

Additional session for Project Presentations: Thu, April 22, 9am-12pm

Project report due on Monday, April 26

 

 

Additional reading

 

 

 

Other resources

 

Books: