🌀 Shannon Entropy Calculator
Measure the information content (uncertainty) of a distribution in bits, for either an equally-likely set of outcomes or a two-outcome probability.
What Shannon entropy measures
Shannon entropy quantifies the average amount of uncertainty, or “information content,” in a random variable’s outcomes. The higher the entropy, the less predictable the outcome is on average. Entropy is measured in bits when using base-2 logarithms, which is the standard convention in information theory.
The formulas used
For a set of N equally likely outcomes (a uniform distribution), entropy is maximized and equals H = log2(N) bits. For a binary (two-outcome) distribution with probability p for one outcome and (1-p) for the other, entropy is the binary entropy function: H(p) = -p·log2(p) – (1-p)·log2(1-p), which peaks at exactly 1 bit when p = 0.5 and falls to 0 bits when the outcome is certain (p = 0 or p = 1).
- Entropy is 0 whenever an outcome is completely certain — there’s no uncertainty left to measure.
- Shannon entropy underlies data compression limits, coding theory, and measures of randomness or diversity in many fields.
This calculator is for general educational purposes.