Information is surprise: an outcome with chance p is worth log₂(1/p) bits. Entropy is the average surprise of a distribution; cross-entropy is the average surprise when you expect Q but the world follows P; and KL divergence is the difference, the price of the wrong belief.