DS 4400: Machine Learning and Data Mining I

Spring 2019

Class Information

Calendar

Additional Reading

Other Resources

 

Instructors:

  • Instructor: Alina Oprea (alinao)
  • TA: Ewen Wang

Class Schedule:

·       Tuesday 11:45am-1:25pm; Thursday 2:50-4:30pm

·       Location:

Office Hours:

·       Alina: Thursday, 4:30-6:00pm, ISEC 625

·       Ewen:  Monday, 5:30-6:30pm, ISEC TBD

Class forum:  Piazza

Class description:

Machine learning is a fast-pacing and exciting field achieving human-level performance in tasks such as image classification, speech recognition. machine translation, precision medicine, and self-driving cars. Machine learning has already impacted greatly our daily lives and has the potential to transform the world even more in the near future. This course will provide a broad introduction to machine learning and cover the fundamental algorithms for supervised and unsupervised learning. We will cover topics related to regression, classification, deep learning, dimensionality reduction, and clustering. The class will also provide an introduction into adversarial machine learning, an emerging area that studies the fundamental security issues of machine learning,

 

Pre-requisites:

·       Probability

·       Statistics

·       Linear algebra

 

Textbook

[ISL] Gareth JamesDaniela WittenTrevor Hastie and Robert Tibshirani. An Introduction to Statistical Learning with Applications in R. [PDF]

Grading

The grade will be based on:

 

-       Assignments – 25%

-       Final project report and presentation – 35%

-       Exam – 35%

-       Class participation – 5%

 Calendar (Tentative)

 

Unit

Week

Date

Topic

Readings

Introduction

1

Tue

01/08

Course outline (syllabus, grading, policies) [PDF]

[ISL] Chapter 1

Review

 

Thu

01/10

Supervised vs unsupervised learning [PDF]

Probability review.

Probability review from Stanford

Linear regression

2

Tue

01/15

Linear regression

Simple linear regression. [PDF]

[ISL] Chapters 3.1 and 3.2

Linear algebra review from Stanford

HW 1 out

Thu

01/17

Multiple linear regression [PDF]

 

3

Tue

01/22

Gradient descent for linear regression [PDF]

Linear regression Stanford notes. Part I

Classification

Thu

01/24

Regularization.

Lasso and ridge regression.

k-Nearest Neighbors (kNN). [PDF]

[ISL] Chapter 6.2

HW 1 due

 

4

Tue

01/29

Cross-validation

Linear classification. Perceptron.

Logistic regression [PDF]

[ISL] Chapter 5.1

[ISL] Chapters 4.1-4.3

Logistic Regression Stanford notes. Part II

 

 

Thu

01/31

Evaluation of ML, metrics

MLE Estimator.

Gradient Descent for Logistic Regression.[PDF]

[ISL] Chapter 4.4

HW 2 out

 

5

Tue

02/05

Metrics, ROC curves

Cross validation

LDA [PDF]

[ISL] Chapter 6.1,

Thu

02/07

Feature selection.

Decision trees [PDF]

[ISL] Chapter 8.1.2

HW2 due 02/08

 

6

Tue

02/12

Information Gain

Decision trees [PDF]

HW3 out 02/12

 

Thu

02/14

Ensemble learning

Random forests

Bagging [PDF]

[ISL] Chapter 8.2

7

Tue

02/19

Boosting (AdaBoost)

[PDF]

 

Thu

02/21

Density estimation.

Naïve Bayes. [PDF]

HW3 due 02/22

 

8

Tue

02/26

SVM

Maximum margin classifier.

Kernels [PDF]

 [ISL] Chapter 9

Project proposal due 02/26

 

 

 

Thu

02/28

 

Class canceled

 

 

Tue

03/05

Spring Break

 

 

 

 

Thu

03/07

 

Spring Break

 

 

9

Tue

03/12

Neural networks and deep learning. [PDF]

 

 

Thu

03/14

Feed-Forward Networks. Forward Propagation. [PDF]

Stanford Deep Learning notes.

HW 4 out

 

10

Tue

03/19

Feed-Forward Networks.

Multi-class classification. Learning Boolean functions [PDF]

Deep learning

Thu

03/21

Convolutional neural networks. [PDF]

 

11

Tue

03/26

Review and exam preparation

Project milestone due on 03/25

 

 

Thu

03/28

Exam

 

 

12

Tue

04/02

Backpropagation. [PDF]

.

HW 4 due

Unsupervised learning

Thu

04/04

Recurrent neural networks

PCA

Clustering: k-means [PDF]

[ISL] Chapters 10.3 and 10.4

Adversarial ML

13

Tue

04/09

Adversarial machine learning [PDF]

 

Thu

04/11

Project presentations

 

 

14

Tue

04/16

Project presentations

 

 

 

TBD

Project report

 

 

 

 

Additional reading

 

 

 

Other resources

 

Books: