- 5/7 - 5/14
- Week 1 / Module 1: Intro, Decision Tree
- Course Map
- Topics:
- Administrative
- Intro to ML, Matrix Data
- Rule-based Classifiers
- Decision/Regression Trees
- Linear Regression
- Background:
- Probabilities
- Convexity1
- Convexity2
- Linear algebra
- Programming: MATLAB, Java, Python, R
- DHS 8.1-8.5
- Other Decision Tree notes
- KMPP ch 16.2
- HW 1
- Due: 5/20
- 5/14 - 5/21
- Week 2 / Module 1: Intro, Regression
- Course Map
- Topics:
- Setup, Cross Validation
- Error, Accuracy, ROC, AUC
- Notes: Linear regression
- Slides
- Screencast 5/14 Lecture Notes 5/14
- Screencast 5/16 Lecture Notes 5/16
Notes: Ridge Regression (normal equations)
- DHS ch 5 (up to 5.4.2)
- KMPP ch 7 (up to 7.5)
- L1-regularization for Regression
- 5/21 - 5/28
- Week 3 / Module 2: Gradient Descent
- Course Map
- Topics:
- Gradient Descent
- Linear Regression with GD
- Logistic Regression
- Newton Method
- DHS ch 5
- KMPP ch 7, 8
- HW2A
- Due: 6/3
- 5/28 - 6/4
- Week 4 / Module 2: Gradient Descent
- Course Map
- Topics:
- Perceptrons
- Neural Networks
- Notes: Perceptrons, Neural Networks
- Screencast 5/28: Perceptron
- Lecture 05/28: Perceptron
- Screencast 5/30 Neural Networks (1)
- Lecture 5/30: NN Basics
- Book: Intro to Neural Networks
- Visualizer : Neural Networks
- Screencast 6/4 Neural Networks (2)
- Lecture 6/4: AutoEncoderNN, Word Vectors_NN, Activation
- Activation Function Table
- DHS ch 6
- HW2B
- Due: 6/17
- 6/4 - 6/11
- Week 5 / Module 3: Generative
- Course Map
- Topics:
- Probabilities as data densities
- Maximum Likelihood, fit params to data
- Gaussian Discriminant Analysis
- Naive Bayes
- Probabilities Primer, Bayesian statistics
- Slides: Generative Intro
- Notes: GDA, Naive Bayes - Andrew Ng
- Slides: GDA
- Screencast 6/6: GDA
- Lecture 6/6: GDA
- DHS ch 2, 3
- KMPP ch 2, 3, 4
- HW 3
- Due: 6/30
- 6/11 - 6/18
- Week 6 / Module 3: Generative
- Course Map
- Topics:
- EM algorithm for fitting mixtures
- Graphical Models
- KMPP ch 20, KMPP ch 11(EM)
- EM code example
- Wikipedia: EM notes
- Notes (Berkeley): EM
- Notes (other): EM
- Slides (set 1): Other EM
- Graphical Models - Bishop chapter
- Graphical Models Bishop video lecture: Part 1, Part 2, Part 3
- 6/18 - 6/25
- Week 7 / Module 4: Boosting
- Course Map
- Topics:
- Online Learning
- Adaboost Algorithm
- Bagging
- RankBoost, Gradient Boosting
- DHS ch 9
- KMPP ch 16
- Adaboost testing error proof
- Paper: Gradient Boosting VS Adaboost
- AdaBoost/Hedge Theory paper
Boosting preferences - rankboost- LambdaMart Slides, JForests Paper, implementation
- HW 4
- Due: 7/12
- 6/25 - 7/2
- Week 8 / Module 4: Boosting
- Course Map
- Topics:
- Active Learning and VC Dimension
- Multiclass ECOC
- Screencast 6/28: Boosting (2), ECOC
- Lecture 6/28 : ECOC, Multiclass
- Notes: Active Learning
- Slides: Active Learning, Screencast
- Wikipedia: VC dimension
- Slides: VC dimension
- Slides:Multiclass ECOC
- Paper: Multiclass ECOC, Screencast
- DHS ch 9.5
- KMPP ch 16.6
- KMPP ch 27.6.2
- 7/2 - 7/9
- Week 9 / Module 5: Features, Margins
- Course Map
- Topics:
- Margins, Boosting Feature Analysis
- PCA and LDA, Lagrangian Multipliers
- Regularized Regression RIDGE and LASSO
- Missing Values
Independence Day : no class/OH
July 4
- Screencast 7/1: Margins, PCA
- Lecture 7/1 : Margins, PCA
- Margins and Feature Analysis for Boosting (screencast)
Feature selection. LASSO / RIDGE regularization for Regression (slides Haar Features for images - Wikipedia ( slides, Optional: Features for text: indexing, ngram matching
Optional: CRFs for text
Missing Values and Naive Bayes
optional: Fischer LDA
Slides: tSNE / paper / implementation
optional: tSNE gradient calculation
- PCA: DHS ch 10
- KMPP ch 25
- 7/9 - 7/16
-
- Week 11 / Module 6: SVM
- Course Map
- Topics:
- Support Vector Machines
- Duality with KKT conditions
- Maximizing Margins Constrained Optimization
- SMO Algorithm
- Notes:Support Vector Machines
- Book Ch: Support Vector Machines
- Convex Optimization
- Slides: Constrained Optimization
- Lecture 7/9-11 : SVM (1,2)
- Screencast 7/9: Intro to SVM, Kernels (1)
- Screencast 7/11: SVM (2)
- Simplified SMO (Stanford)
- John Platt SMO
- DSH ch 5.11
- KMPP ch 14
- HW 5
- Due: 7/25
- 7/16 - 7/23
- Week 12 / Module 6: SVM, Kernels
- Course Map
- Topics:
- Kernels
- Kernels for SVM
- K-Nearest Neighbor
- Kernel Similarity and KNN
- Kernel Density Estimation
- Heat Kernels, Harmonic Equation
- Kernels of other Algorithms, representer theorem
- Instance-based classification
- Local Classifiers
- Lecture 7/9-16 : kernels (1,2)
- Screencast 7/16: Kernels (2)
- Lecture 7/18 : kernels (3), KNN, t-SNE
- Screencast 7/18: KNN, Kernel-PCA
paper: Kernel Methods in Machine Learning
- 7/23 - 7/30
- Week 13 / Module 7: Adv NN
- Course Map
- Topics:
- Adv NN
- RNN
- TRN
- HW6
- Due: 8/11
- 7/30 - 8/6
- Week 14 / Module 7: Adv NN
- Course Map
- Topics:
- TRN, attention
- Lecture 7/30 : Recurrent NNets, LSTM, GRU (d2l slides w/ annotations)
- Screencast 7/30: Recurrent NN
- RNN (d2l html+ pytorch)
- Lecture 8/1 : RNN : Encoder-Decoder, Seqence-to-Sequence, Machine Translation (d2l slides w/ annotations)
- Screencast 8/1: RNN: GRU, LSTM, Encoder-Decoder for Seq-to-Seq
- RNN-Modern (d2l html+ pytorch)
- 8/8
- Week 14 Project
Project Report due Fri 8/8
- 8/6 - 8/13
- Week 14 / Module 7: Adv NN
- Course Map
- Topics:
- TRN, attention