DS 4400: Machine Learning and Data Mining I

Fall 2018

Class Information

Calendar

Additional Reading

Other Resources

 

Instructors:

  • Instructor: Alina Oprea (alinao)
  • TA: Anand Lad

Class Schedule:

·       Tuesday 11:45am-1:25pm; Thursday 2:50-4:30pm

·       Location: Ryder Hall 158

Office Hours:

·       Alina: Thursday, 4:30-6:00pm, ISEC 625

·       Anand: Tuesday, 2-3pm, ISEC 605

Class forum:  Piazza

Class description:

Machine learning is a fast-pacing and exciting field achieving human-level performance in tasks such as image classification, speech recognition. machine translation, precision medicine, and self-driving cars. Machine learning has already impacted greatly our daily lives and has the potential to transform the world even more in the near future. This course will provide a broad introduction to machine learning and cover the fundamental algorithms for supervised and unsupervised learning. We will cover topics related to regression, classification, deep learning, dimensionality reduction, and clustering. The class will also provide an introduction into adversarial machine learning, an emerging area that studies the fundamental security issues of machine learning,

 

Pre-requisites:

·       Probability

·       Statistics

·       Linear algebra

 

Textbook

[ISL] Gareth JamesDaniela WittenTrevor Hastie and Robert Tibshirani. An Introduction to Statistical Learning with Applications in R. [PDF]

Grading

The grade will be based on:

 

-       Assignments – 20%

-       Final project report and presentation – 25%

-       Midterm exam – 25%

-       Final exam – 25%

-       Class participation – 5%

 Calendar (Tentative)

 

Unit

Week

Date

Topic

Readings

Introduction

1

Thu

09/06

Course outline (syllabus, grading, policies)

[PDF]

[ISL] Chapter 1

Review

2

Tue

09/11

Probability review.

Linear algebra review. [PDF]

Probability review from Stanford

Linear algebra review from Stanford

 

Supervised learning

Thu

09/13

Linear regression

Simple linear regression. [PDF]

[ISL] Chapters 3.1 and 3.2

 

3

Tue

09/18

Multiple linear regression

Gradient descent. [PDF]

Linear regression Stanford notes. Part I

HW 1 out

 

 

Thu

09/20

Gradient descent for linear regression

Regularization.

Lasso and ridge regression. [PDF]

[ISL] Chapter 6.2

 

4

Tue

09/25

Linear classification. Perceptron.

k-Nearest Neighbors (kNN).

Cross validation. [PDF]

[ISL] Chapter 5.1

 

 

Thu

09/27

Logistic regression

MLE Estimator.

Evaluation of ML, metrics [PDF]

[ISL] Chapters 4.1-4.3

Logistic Regression Stanford notes. Part II

HW 1 due: Friday, Sept. 28

5

Tue

10/02

Gradient Descent for Logistic Regression.

LDA.

Feature selection. [PDF]

[ISL] Chapter 4.4

 

 

Thu

10/04

Metrics, ROC curves.

Feature selection.

Decision trees.

Information Gain. [PDF]

 

[ISL] Chapter 6.1,

HW2 out

 

6

Tue

10/09

Decision tree ID3; pruning.

Ensembles

Bagging [PDF]

[ISL] Chapter 8.1.2

Thu

10/11

Ensemble learning

Random forests

Boosting (AdaBoost) [PDF]

[ISL] Chapter 8.2

 

7

Tue

10/16

Midterm exam

 

 

Thu

10/18

SVM

Maximum margin classifier. [PDF]

[ISL] Chapter 9

HW 2 due: Oct. 19

Deep learning

8

Tue

10/23

SVM. Kernels

Density estimation. [PDF]

Project proposal due: Oct 24

 

Thu

10/25

Naïve Bayes.

Metrics for classification. [PDF]

 

 

 

9

Tue

10/30

Neural networks and deep learning.

Feed-Forward Networks. [PDF]

Stanford Deep Learning notes.

HW 3 out: Oct 31

Unsupervised learning

Thu

11/01

Convolutional neural networks [PDF]

 

 

10

Tue

11/06

Backpropagation.

Convolutional neural networks. [PDF]

 

 

 

Thu

11/08

Backpropagation, cont.

Regularization. [PDF]

 

11

Tue

11/13

Recurrent neural networks.

Dimensionality reduction

PCA, Auto-encoders [PDF]

HW3 due: Nov 14

[ISL] Chapters 10.1 and10.2

 

Thu

11/15

Clustering

k-means

Hierarchical clustering [PDF]

[ISL] Chapters 10.3 and 10.4

Adversarial machine learning

12

Tue

11/20

Adversarial machine learning

Project milestone

 

Thu

11/22

Thanksgiving

No class

 

 

13

Tue

11/27

CLASS CANCELLED

 

Review

Thu

11/29

Review [PDF]

 

Project presentations

14

Mon

12/03

Project presentations, 3-5:30pm

ISEC 655

 

 

 

Tue

12/11

Final exam, 2-5pm

ISEC 655

Project report: due Friday, Dec 7

 

 

 

Additional reading

 

 

 

Other resources

 

Books: