HW3 (due Sept 20, 2007)

1. Products. Let X be a random variable taking values 1,2, and 3. And p_X(1)=1/2, p_X(2)=1/4, and p_X(3)=1/4. Let X₁, X₂, ... be drawn i.i.d. according to the distribution of X. Find the limiting behavior of the product (X₁ X₂ ... X_n)^1/n. (From Cover and Thomas 2nd edition, 3.8)

2. True or false. Determine if the following is true or false. Give a proof (or argument) if so, otherwise, show a counter example.
a. H(XY|Z) ≤ H(X|Z)+H(Y|Z)
b. H(X+Y) = H(X) + H(Y)
c. H(X₁X₂...X_n)/n ≤ H(X₁X₂...X_n-1)/(n-1), where X_n is the output at the nth instance from a stationary random process.