Ergodic Theory

The mathematics came from real analysis. Lebesgue's measure theory could say precisely what "almost every trajectory" and "average over all states" mean. In 1931 John von Neumann proved that time averages converge, in an averaged sense. George David Birkhoff, told of the result, proved within weeks that they converge along almost every individual trajectory, and published first. For an ergodic system, one that cannot be split into two separate invariant parts, the time average then equals the space average. Boltzmann's assumption had become a theorem with precise conditions. Checking those conditions for a real gas is another matter, and for most physical systems it is still open.

In 1958 Andrey Kolmogorov brought in Shannon's information theory. He defined the entropy of a dynamical system: the rate at which watching it reveals new information. Yakov Sinai gave the definition its final form, and in 1970 Donald Ornstein showed that for random processes like coin tossing, entropy is the whole story. Processes with the same entropy are equivalent. In 1970 Sinai proved that a ball bouncing among convex obstacles, a fully deterministic system, is ergodic and chaotic, and such billiards were later shown to be, in Ornstein's sense, exactly as random as tossing a coin.

List the powers of two: 2, 4, 8, 16, 32, 64, 128, 256, 512, 1024, … and look at their first digits: 2, 4, 8, 1, 3, 6, 1, 2, 5, 1, … How often is the first digit a 1?

A number starts with the digit 1 exactly when the fractional part of its base-10 logarithm lies between $0$ and $\log_{10} 2 \approx 0.301$. For $2^n$, the logarithm is $n \log_{10} 2$. So the question is how often the fractional part of $n \times 0.30103\ldots$ lands in the interval $[0, 0.301)$. For $2^{10} = 1024$: $10 \times 0.30103 = 3.0103$, with fractional part $0.0103$, inside the interval.

Adding $\log_{10} 2$ each time and keeping the fractional part is a rotation of a circle of circumference 1 by the angle $0.30103\ldots$ Because $\log_{10} 2$ is irrational, the rotation never repeats, and it is ergodic. Hermann Weyl and others proved between 1909 and 1916 that the orbit of such a rotation spends time in each arc in proportion to its length, and ergodic theory later explained why: an irrational rotation is ergodic in the strongest possible way, with only one invariant measure. So the first digit is 1 with long-run frequency $\log_{10} 2 \approx 30.1\%$.

Counting confirms it: of $2^1$ to $2^{1000}$, exactly 301 begin with 1. The same argument gives frequency $\log_{10}(1 + 1/d)$ for first digit $d$. Digit 7 should appear about 5.8% of the time, and it does, 56 times in the first thousand, although it first appears only at $2^{46} = 70{,}368{,}744{,}177{,}664$. This is Benford's law, and ergodic theory explains why it holds exactly for the powers of two. Auditors use Benford's law to detect fabricated accounts, whose first digits tend to be too evenly spread.

In 1977 Hillel Furstenberg showed that ergodic theory could prove deep results about whole numbers. He translated Szemerédi's theorem on arithmetic progressions, from extremal combinatorics, into a statement about recurrence, and proved that. Since then, the exchange between dynamics and number theory has become one of the most productive in mathematics. Its limits are sharply marked by Furstenberg's own question from 1967: whether multiplying by 2 and by 3 together leaves any exotic statistics unchanged. It is still open.