CSG195 08F: Syllabus

Note: This is an approximate syllabus; it may change at any time.

1. Lecture 01, Fri, Sep 12 2008
   • Admistrivia
   • Introduction to Information Theory and its applications

2. Lecture 02, Tue, Sep 16 2008
   • Probability primer I

3. Lecture 03, Fri, Sep 19 2008
   • Probability primer II
   • Homework 01 assigned

4. Lecture 04, Tue, Sep 23 2008
   • Entropy and its properties
   • Reading: Cover and Thomas 2.1-2

5. Lecture 05, Fri, Sep 26 2008
   • Conditional entropy, relative entropy, mutual information
   • Reading: Cover and Thomas 2.2-4
   • Homework 01 due
   • Homework 02 assigned

6. Lecture 06, Tue, Sep 30 2008
   • Chain rules, data processing inequality, Fano's inequality
   • Reading: Cover and Thomas 2.5-6, 2.8, 2.10

7. Lecture 07, Fri, Oct 03 2008
   • Markov chains, entropy rate of stochastic processes
   • Reading: Cover and Thomas 4
   • Homework 02 due
   • Homework 03 assigned

8. Lecture 08, Tue, Oct 07 2008
   • Compression I: codes and decodability, Kraft's inequality, bounds on optimal codes
   • Reading: Cover and Thomas 5.1-4

9. Lecture 09, Fri, Oct 10 2008
   • Compression II: more Kraft's inequality, block coding, Huffman codes
   • Reading: Cover and Thomas 5.5-8
   • Homework 03 due
   • Homework 04 assigned

10. Lecture 10, Tue, Oct 14 2008
    • Compression III: twenty questions, arithmetic coding, randomness
    • Reading: Cover and Thomas 5.7, 5.10, 5.11, 13.3

11. Lecture 11, Fri, Oct 17 2008
    • Asymptotic Equipartition Property (AEP) and its consequences
    • Reading: Cover and Thomas 3

12. Lecture 12, Tue, Oct 21 2008
    • Information theory and statistics: the method of types and applications
    • Reading: Cover and Thomas 12.1-2

13. Lecture 13, Fri, Oct 24 2008
    • Lempel-Ziv, universal source coding
    • Reading: Cover and Thomas 13.4
    • Homework 05 due
    • Homework 06 assigned

14. Lecture 14, Tue, Oct 28 2008
    • Channel coding
    • Reading: Cover and Thomas 7

15. Lecture 15, Fri, Oct 31 2008
    • The maximum entropy method
    • Reading: Cover and Thomas 12.1-2
    • Homework 06 due

16. Remainder of the term...
    • Applications of information theory: lectures and student presentation

Switch to:

• CSG195 08F Home Page