CSG399 05F: Homework 02

Last modified:

General Instructions

1. Feel free to work with others on this assignment. However, you must acknowledge with whom you worked, and you must write up your own solutions.

2. All of the problems below are from the text, Cover and Thomas, or refer to problems from the text.

Problems

1. CT, Chapter 2, Problem 1.
2. Repeat CT, Chapter 2, Problem 1, part (a) for a biased coin where the probability of heads is p.
3. Prove that the definition of entropy satisfies the "grouping" property, as defined in class and given in CT, Chapter 2, Problem 4.
4. CT, Chapter 2, Problem 19.

   Note: 2^H(X) is referred to as the perplexity of X. It is a measure of the "effective" size of the range of X, with respect to the underlying distribution over this range.

5. CT, Chapter 2, Problem 28.

   Hint: You will likely make use of a convexity argument. See CT Section 2.6 for the definition of convexity; we will cover this in class as well.