Differential Equations

Leonhard Euler turned the tricks into methods. In 1743 he showed that every linear equation with constant coefficients, the kind that describes springs, pendulums and circuits, reduces to finding the roots of a polynomial. He classified equations by type and solved type after type. And where no method worked, he proposed computing the solution approximately: start at the known point, follow the current slope for a short step, recompute the slope, and repeat.

The hope that every equation could be solved by formula ended with Joseph Liouville. In 1841 he proved that even the innocent-looking Riccati equation $y' = x^2 + y^2$ has no solution built from elementary functions and integrals. It became clear that solvable equations were the exception.

That raised a basic question: if no formula exists, does a solution exist at all? Augustin-Louis Cauchy proved in the 1820s that it does, under mild conditions, and that it is unique. Rudolf Lipschitz and Émile Picard later gave the sharp conditions and the proof by successive approximation. Sofia Kovalevskaya extended existence to partial differential equations in her thesis of 1874. Determinism, the idea that the present fixes the future, was now a theorem.

The simplest growth law is $y' = y$: the rate of growth equals the current amount, starting from $y(0) = 1$. Its exact solution is $y = e^t$, so at $t = 1$ the answer is $e = 2.71828\ldots$ Pretend we don't know that, and use Euler's method.

With step size $h$, each step multiplies the current value by $(1 + h)$, since the value grows by $h$ times itself. Reaching $t = 1$ takes $1/h$ steps, so the estimate is $(1 + h)^{1/h}$:

Every tenfold decrease in step size cuts the error roughly tenfold. The method converges, but slowly: the error is proportional to $h$. That is why practical solvers use cleverer schemes, such as the fourth-order Runge–Kutta method, which samples the slope four times per step and whose error shrinks like $h^4$, ten thousand times smaller for each tenfold cut in step size.

The example also shows compound interest. Growing 100% a year, compounded $n$ times, gives $(1 + 1/n)^n$, which approaches $e$. Jacob Bernoulli found the number this way in 1683. Euler's method and compound interest are the same calculation.

By the late nineteenth century the question had shifted from "what is the solution?" to "what does it do?" Will a planet's orbit stay bounded? Will a population settle down or oscillate? Henri Poincaré answered such questions from the geometry of the equations alone, founding the theory of dynamical systems. One of the oldest questions of that kind, how many isolated periodic cycles a polynomial equation in the plane can have, was Hilbert's sixteenth problem in 1900, and it is still open.