This course covers techniques for analyzing very large data sets. We introduce the MapReduce programming model and the core technologies it relies on in practice, such as a distributed file system. Related approaches and technologies from distributed databases and Cloud Computing will also be introduced. Particular emphasis is placed on practical examples and hands-on programming experience. Both plain MapReduce and database-inspired advanced programming models running on top of a MapReduce infrastructure will be used.
Acknowledgment: This course was kindly supported by an AWS in Education Grant award from Amazon.com, Inc.
(Future lectures and events are tentative.)
Overview: data and harware trends
Scalability and metrics
Google File System, Hadoop's HDFS
|Assignment 1 out. Due Feb 1.|
|Jan 27||MapReduce and Hadoop|
(Includes: in-mapper combining, sorting, secondary
|Assignment 2 out. Due Feb 15.|
(Includes: order inversion, per-record computation,
group-by, global counters, random sampling and shuffling, quantiles,
Basic Algorithms: Advanced (Includes: reduce-side join, replicated join,
semi-join with Bloom filter)
|Assignment 3 out. Due Mar 1.|
|Feb 24||Pig and Pig Latin|
|Mar 3||Relational Databases||Assignment 4 out. Due Mar 22.|
|Mar 10||No class. Spring Break.|
|Project starts: team forming, proposal. Due Mar 29.|
|Mar 24||Midterm exam|
(Includes: single source shortest path, PageRank)
|Project progress report assignment out. Due Apr 12.|
(Includes: Pairs and Stripes, theta-join)
|Apr 14||Data Mining 1: clustering, classification||Project final report assignment out. Due Apr 26.
Project presentation assignment out. Due Apr 27.
|Apr 21||Data Mining 2: ensemble methods, regression, matrix manipulation|
|Apr 28||Project presentations||Same time and location as lecture.|
Instructor: Mirek Riedewald
Meeting times and location: check registrar system for up-to-date info
CS 5800 or CS 7800, or consent of instructor
Safari Books Online at NEU: http://proquest.safaribooksonline.com.ezproxy.neu.edu/ (might have changed in the meantime)
A commitment to the principles of academic integrity is essential to the mission of Northeastern University. The promotion of independent and original scholarship ensures that students derive the most from their educational experience and their pursuit of knowledge. Academic dishonesty violates the most fundamental values of an intellectual community and undermines the achievements of the entire University.
For more information, please refer to the Academic Integrity Web page.