How to Prepare for the Course?
This class addresses theoretical and practical aspects of machine learning in a balanced way. This means, we will focus on understanding learning frameworks, the concepts behind optimization algorithms and in several cases we will implement various learning algorithms and investigate how they perform on different tasks. While officially the pre-requisite courses for this class is only Algorithms, I expect that each student has familiarity with college-level calculus, linear algebra, and probability theory. These include differentiation, integration, vector spaces and matrices, linear independence, solving linear systems, basic probabilistic definitions and manipulations (e.g. random variable, Bayes’ rule, sum and product rule), probability distributions, expectations and variance, and so on. We will briefly review these concepts when we need them, but you should not expect me to discuss them at length even if you haven't seen them before.
Here's what I think is useful to prepare for the class, though certainly not the only way. Of course, you've probably seen these topics before and you may only need quick reminders.
Useful books for understanding probabilities
R. M. Gray, L. D. Davisson. Introduction to statistical signal processing, 2011.
Chapters 1-4. The book is generally above the level needed in CS 6140.
Larry Wasserman's book All of Statistics (2005) contains the right material and is of good quality.
A useful book for understanding linear algebra
G. Strang. Introduction to Linear Algebra. Wellesley-Cambridge Press, 2009.
Chapters 1-4.
Highly recommended video lectures.
A useful book on calculus
G. Strang. Calculus. Wellesley-Cambridge Press, 2010.
Old edition of the book (1991) available here.
Highlights of calculus video lectures.
I haven't had a chance to read this book and but I saw the lectures.
Practicing scientific writing in Latex
Class assignments will be expected to be typed. It will thus come handy to be familiar with text editors that can handle formulas in a professional manner. I recommend Lyx (beginners) and TeXnicCenter (advanced; Windows only). Both will ask you to have a Latex compiler installed (I use MiKTeX; both Lyx and TeXnicCenter will install it for you if you have Windows). Note that I recommend TeXnicCenter 1.0 Stable RC1 version (the newer ones were not very good when I tested them). For a Mac, I recommend TeXShop. Or you can use Overleaf on the web.
Last updated: May 12, 2024