# Entropy

Reading about mutual information between amino acids in protein-protein interaction pairs, and got distracted:

Although not particularly obvious from this equation, $H(X)$ has a very concrete interpretation: Suppose $x$ is chosen randomly from the distribution $P_X(x)$, and someone who knows the distribution $P_X(x)$ is asked to guess which $x$ was chosen by asking only yes/no questions. If the guesser uses the optimal question-asking strategy, which is to divide the probability in half on each guess by asking questions like “is $x$ greater than $x_0$?”, then the average number of yes/no questions it takes to guess $x$ lies between $H(X)$ and $H(X)+1$ (Cover and Thomas, 1991). This gives quantitative meaning to “uncertainty”: it is the number of yes/no questions it takes to guess a random variables, given knowledge of the underlying distribution and taking the optimal question-asking strategy.

Scholarpedia: Mutual information