Entropy and the Second Law

The corner of the box#

Take a sealed box, and put every gas molecule in it into one corner. Let go.

You know exactly what happens. The gas fills the box. You have never once seen the reverse — a room's worth of air quietly collecting itself in one corner, leaving you gasping. And yet: nothing in the physics of the individual collisions forbids it.

Play the film of two molecules bouncing off each other backwards and you get another perfectly legal collision. Newton's laws are time-symmetric. So are Maxwell's equations, and so is the Schrödinger equation. If you filmed the gas spreading out and ran the film in reverse, every single frame would obey the microscopic laws exactly. The reversed film is dynamically legal. It just never happens.

This gap — between reversible microscopic laws and a wildly irreversible world — is the puzzle that entropy solves.

Microstates and macrostates#

The resolution comes from being careful about what you actually measure.

A microstate is the complete specification of the system: the position and momentum of every single molecule. A macrostate is what you can measure from outside: pressure, temperature, and "how much gas is in the left half."

The crucial point is that a huge number of different microstates all look like the same macrostate. If you ask only "how many molecules are on the left?", then swapping molecule 17 with molecule 4,000,922 gives a completely different microstate but an identical macrostate. Macrostates are coarse; microstates are fine.

Ludwig Boltzmann's insight, carved on his gravestone in Vienna, is that entropy is simply a count of that multiplicity:

$$S = k_B \ln W$$

Here $W$ is the number of microstates consistent with the macrostate, and $k_B = 1.380649 \times 10^{-23}\ \mathrm{J/K}$ is Boltzmann's constant, which exists only to convert a pure count into conventional units of joules per kelvin. The logarithm is there so that entropy adds: two independent systems have $W_1 W_2$ combined microstates, and $\ln(W_1 W_2) = \ln W_1 + \ln W_2$.

That is the whole definition. Entropy is not a substance, not a force, and not a measure of how untidy something looks. It is the logarithm of how many ways the world could be arranged and still look the way it does.

Watching a macrostate settle#

Every particle starts crammed into the bottom-left corner. Hit play and watch two things at once.

Watch an individual particle: it just bounces. It has no preference for the right-hand side, no memory of where it started, and it is as likely to head back left as to continue right. Now watch the counters at the top. The left/right split climbs off 160 | 0 and settles near 50/50, and the entropy readout climbs with it and then stops climbing.

Two details are worth pausing on. First, the split never freezes at exactly half — it jitters. That jitter is a real fluctuation, an entropy decrease, happening constantly. Second, turn the particle count down to 20 and watch the jitter get proportionally much bigger; turn it up to 400 and the split becomes glued to the middle. The second law gets stricter as systems get bigger, which is a strange property for a law of nature to have — and a clue about what kind of law it really is.

Press Reset to put every particle back in the corner. Note what you had to do: you had to intervene from outside. The system will not do it for you.

The combinatorics behind it#

Strip the box down to its bookkeeping. Each of $N$ particles is either left or right — one bit each, so there are $2^N$ microstates in total, all equally likely if the dynamics doesn't play favourites. The macrostate "exactly $k$ particles on the left" is realized by

$$W(k) = \binom{N}{k} = \frac{N!}{k!\,(N-k)!}$$

microstates, so by Boltzmann's formula its entropy is $S(k) = k_B \ln \binom{N}{k}$.

Only one microstate has all $N$ particles on the left, so that macrostate has $W = 1$ and $S = 0$. The balanced macrostate has $\binom{N}{N/2}$ microstates, which for large $N$ is close to the entire $2^N$ total. The ratio is the whole story:

$$\frac{W(\text{all left})}{W(\text{half and half})} \approx \sqrt{\frac{\pi N}{2}} \cdot 2^{-N}$$

For a mole of gas, $N \approx 6 \times 10^{23}$, and $2^{-N}$ is a number with more zeros after the decimal point than there are atoms in the observable universe. Using Stirling's approximation $\ln N! \approx N\ln N - N$, the maximum entropy of the two-box split works out to the clean result

$$S_{\max} = k_B \ln 2^N = N k_B \ln 2$$

which is exactly one bit of entropy per particle — the information you'd need to say which side each molecule is on.

Slide $N$ upward and watch the shape change. At $N = 8$ (the faint violet curve, kept for comparison) the distribution is a broad hump: a 6-2 split is genuinely common, and even 8-0 turns up once in 256 tries. By $N = 40$ the shoulders have pulled in. By $N = 200$ the curve is a spike.

The number to track is the peak width in the corner: it shrinks as $1/\sqrt{N}$. Absolute fluctuations do grow — a bigger box has bigger raw swings, going as $\sqrt{N}$ — but the fraction of the population involved shrinks. At $N = 100$ you expect swings of a few percent. At $N = 10^{23}$ the relative swing is around $10^{-12}$, which is why a barometer never twitches. Extrapolate the curve you're sliding and you have the quantitative version of "never re-gathers": not forbidden, just a spike so narrow that the rest of the axis has no measurable probability at all.

Entropy is not messiness#

The popular gloss — "entropy is disorder" — is a metaphor that has escaped its enclosure, and it misleads in both directions.

It fails on real systems constantly. Cool a supersaturated solution and crystals form: the solute becomes strikingly more ordered to the eye, yet total entropy rises, because the heat released spreads through the solvent and buys back more microstates than the crystal gave up. Hard spheres at high density spontaneously form an ordered crystal lattice for the same reason — the ordered packing gives each sphere more room to wiggle, so the crystal actually has more microstates than the disordered fluid at that density. Entropy chose the tidy-looking state.

It also fails in the other direction. A messy desk and a tidy desk have essentially identical entropy. "Messy" is a judgment about which arrangements you find useful, and the universe has no opinion about that.

The honest statement is that entropy counts microstates per macrostate. "Disorder" is a rough proxy that happens to work for the textbook case of a gas expanding, and fails as soon as energy and volume trade against each other. When in doubt, count.

A second correction is worth making: entropy is not a force. Nothing pushes the gas outward. Every particle in the widget above moves in a straight line until it hits something. "Entropic forces" — the elasticity of rubber, the hydrophobic effect, depletion attraction — are real and useful effective descriptions, but underneath, they are all statistics: the system is not pulled toward high-entropy states, it simply spends almost all of its time there because almost all of its states are there.

The arrow of time#

So where does irreversibility come from, if not from the microscopic laws?

It comes from the initial condition. The dynamics are symmetric; the boundary condition is not. Our universe began in a state of astonishingly low entropy, and everything since has been the unfolding of that improbability. Roger Penrose estimated the required fine-tuning of that initial state at roughly one part in $10^{10^{123}}$ — a number that cannot be written out in a universe this size.

Everything we call the arrow of time is downstream of that. Eggs break and don't unbreak. Coffee cools to room temperature and never spontaneously reheats. Memory works in one direction — recording a memory is itself an entropy-increasing act, which is why you remember the past and not the future. The second law is the only fundamental law of physics that distinguishes past from future, and it does so not by adding a direction to the equations but by noting which end of time we started from.

Clausius stated the law thermodynamically in 1865: $\Delta S \geq 0$ for an isolated system, with equality only for reversible processes. That form is exact for macroscopic systems only because $N$ is enormous. Strictly, the second law is statistical: the fluctuation theorem quantifies exactly how often small systems run backwards, and the predicted violations have been measured directly in optical-tweezer experiments on micron-scale beads. At that size the second law is a suggestion. At human size it is as good as absolute.

Where it shows up#

The reach of this one counting argument is unreasonable.

• Engines. No heat engine between reservoirs at $T_H$ and $T_C$ can beat the Carnot efficiency $\eta = 1 - T_C/T_H$. This is a hard ceiling on every power plant and car engine ever built, and it follows from $\Delta S \ge 0$ alone.
• Refrigerators and life. Both create local order, and both must dump more entropy into their surroundings than they remove. Your body maintains its exquisite internal order by radiating heat and exporting high-entropy waste; the books balance, with room to spare.
• Information. Shannon's entropy $H = -\sum p_i \log p_i$ has the same form as Boltzmann's for a reason. Landauer's principle makes the connection physical: erasing one bit of information must dissipate at least $k_B T \ln 2$ of heat, a bound now confirmed experimentally. Information is physical, and the currency is entropy.
• Chemistry. Reactions run in the direction that lowers the Gibbs free energy $G = H - TS$, which is just the second law applied to a system in contact with a thermal bath. The $-TS$ term is why some reactions that absorb heat still proceed spontaneously.
• Black holes. Bekenstein and Hawking found that a black hole's entropy is proportional to its horizon area, not its volume: $S = k_B A / 4\ell_P^2$. Counting microstates at the largest scale in physics remains one of the sharpest open problems in quantum gravity.

One formula, $S = k_B \ln W$, connects a jar of gas to the efficiency of a turbine, the cost of deleting a file, and the interior of a black hole.