Question: 1. Entropy, conditional entropy, mutual information, information gain I have a very messy sock drawer, containing 10 red socks, 5 blue socks, 4 yellow socks, and 1 black sock. …

Exercise Define a function conditional_entropy that • takes • a distribution p as its first argument, • a conditional distribution q as its second argument, and • returns the conditional entropy H (q|p).

Define a function conditional_entropy that - takes - a distribution p as its first argument, - a conditional distribution q as its second argument, and - returns the conditional entropy H (q∣p).

Question: We would like to compute entropy, joint entropy and conditional entropyusing Python (Matlab, R, or any other programming language of yourchoice).

Question: Problem 2 - Conditional entropy (10 marks) Let X,Y be two random variables with respective sample spaces X,Y and probability distributions p (x),p (y) (x∈X,y∈Y).

Question: The conditional entropy is defined for a conditional distribution q = [q;iliesjetand a distribution p = (pilies as follows: 1 Pi (11) jET where, by convention, values of q/l), and • the …

Question: Definition 6.1 The mutual information between two random variables X and Y is defined as I (X,Y)=H (X)−H (X∣Y) where H (X) is the (discrete) entropy of the random variable X taking …

Question: (Zero conditional entropy) Show that if (x|)=0, then Y is determined by x in the sense that, for every a with Px (a)>0, there is only one ...

(b) Please find the entropy of Y, H (Y)? (c) Please find the joint entropy of H (X,Y)? (d) Please find the conditional entropy of H (YX)? Show transcribed image text

The conditional entropy is defined for a conditional distribution q = [9j|ilies.jet and a distribution p = [pilies as follows: H (q [p) = XP: X) ; log2 9jli 1 IES JET where, by convention, • the summand …