Editorial Remark: This article is based on a lecture delivered by Prof. John D. Barrow at a conference under the MOTIVATE program of the Millennium Mathematics Project. The topic of the conference was "Chaos". The Editorial Board of PhysicaPlus is grateful to Prof. Barrow for his kind permission to reproduce this article. Prof. Barrow succeeds in explaining the notion of chaos, playing an important role in highly complex systems, with non-linear interactions, in a comprehensive, and yet clear and simple manner.

One of the great things about mathematics is that it can tell you things about the world that you could not discover in any other way. The application of mathematics to predict the way that things behave is crucial to our attempts to understand problems like how the weather changes over periods of days or of decades, how ecological balances might be affected by sudden changes in human activity, how economies will change when financial policies are altered, how to optimise the braking system of a car, how to assess risk, how turbulent air flows around an aircraft wing, or how close to Earth an approaching asteroid might pass. In all these problems we are interested in a reliable understanding of a complicated situation and an ability to predict what might happen in the future.

Predictability and stability

In problems like those we have listed, we are interested in arriving by some means at the correct mathematical model for the problem of interest, inputting reliable information about the past and the present state of affairs, so as to extract predictions about the future. Traditionally, mathematics had been applied to a collection of "simple" problems which described situations that are insensitive to small changes. Insensitive situations are characterised by the property that a small change produces a small result. For instance, if the change is just adding 5 to a number then, no matter how uncertain that number is, the uncertainty of the sum after 5 has been added will be the same. A result of this property is that a small uncertainty in your knowledge about the present situation will only produce a correspondingly small uncertainty in your prediction of its future behaviour.

Another example is the pendulum of a clock. When the oscillations of the pendulum have a small amplitude, so that the angle of deviation from the vertical is small enough for sinq to be very well approximated by q, the period of oscillation T of the pendulum depends on the length of the pendulum, L and the acceleration due to gravity, g=9.81ms^-2, via the formula

You notice that T does not depend upon the amplitude of the oscillation or the position from which the pendulum starts swinging. This means that when the clock stops you can rewind it and start the pendulum swinging again by just giving it a little push. It doesn't really matter how you push it, or where you release it from: it will settle down to swing back and forth with period T almost straight away. When the clock weights have descended and given up their gravitational potential energy to sustain the pendulum's motion, the pendulum will stop swinging. This is a typical stable system. Uncertainties in how it starts don't make an important difference in predicting its future.

Another insensitive example is provided by dropping a ball-bearing in a smooth bowl and allowing it to oscillate back and forth. There is no clock weight to keep the motion going in this case and very quickly the oscillations will die away to zero. No matter how you start the ball-bearing moving around inside the bowl, it will eventually come to rest at the lowest point. Displace the ball slightly from this point and it will inevitably return.

In this system any slight uncertainty in the position of the ball becomes less and less important as time goes on. The system possesses what mathematicians call an attractor, to which a wide class of different starting behaviours tend with the passage of time. The ball has reached an unchanging attractor, just like the pendulum does when the clock weights have run down. But attractors can also be periodically changing. After the pendulum first started swinging back and forth, it quickly reached an attractor which displayed periodic oscillations with time period T. This attractor is reached regardless of how the small oscillations began.

A Little Bit of History

Scientists focussed attention upon stable problems like these examples for hundreds of years. They could often be solved exactly using simple mathematics and they allowed approximations to be made (like the assumption that the oscillations of the pendulum have small amplitude) without producing large errors in the predictions that resulted. Constant focus on problems like this had an influence on mathematicians' overall view of the universe. The French mathematician Pierre Laplace (1749-1827) had the following idea. If a "super being" knew the positions and motions of all the particles in the Universe then Newton's laws of motion could be used to predict the future or reconstruct the past exactly. The exact prediction of a future state uniquely and completely from the present is usually called determinism. Here is what Laplace said about his imaginary superbeing:

"An intellect which at any given moment knew all the forces that inanimate nature and the position of the beings that compose it, if this intellect were vast enough to submit its data to analysis, could condense into a single formula the vast movement of the greatest bodies of the universe and that of the lightest atom: for such an intellect nothing could be uncertain; and the future just like the past would be present before its eyes."

The first person to see why there is more to predicting the future than this was the mathematical physicist, James Clerk Maxwell, famous for discovering the laws of electromagnetism ("Maxwell's equations") and the statistical motions of molecules ("the Maxwell distribution"). On 11th February 1871, he gave a talk in Trinity College, Cambridge, one evening to a small group of scientists and non-scientists. They were interested in determinism in the light of Charles Darwin's new theory of evolution, and whether it allowed us to have freewill.

Maxwell talked about something unexpected. The precision of Newton's laws were well known. They gave very accurate descriptions of the motions of the planets and the Moon. But, Maxwell argued, people had been brainwashed into thinking that the world was totally predictable because they only paid much attention to situations where that was true. They thought that just because there were exact mathematical laws of motion that all motion was completely predictable. Yet, there were many situations where a small change produces a large effect. Moreover, a little uncertainty in what the small change is like, will create a correspondingly large uncertainty in its future effect. Of these "unstable" situations, Maxwell said that

"It is manifest that the existence of unstable conditions renders impossible the prediction of future events, if our knowledge of the present state is only approximate."

He suggested that his audience would learn much more important things about Nature if they concentrated their attention upon its instabilities and uncertainties rather than the strict determinism of the laws of motion themselves. Maxwell gave the graphic example of a switch point on a railway track as an example of how a very small change (in the direction of the point lever) produces a large change in the path of the train that results.

Another example of one of these unstable situations is the balancing of the ball-bearing on an inverted bowl. If we put the ball-bearing at the highest point of the inverted bowl then the slightest perturbation will cause it to roll away down the outside of the bowl. Small changes in the direction in which it gets pushed initially, will result in a large changes in its path down the outside of the bowl.

These ideas were stressed again by the French mathematician Henri Poincaré in the period 1890-1900 in connection with his studies of celestial motions of planets in the solar system. He reiterated Maxwell's message by pointing out how

"It may happen that small differences in the initial conditions produce very great ones in the final phenomena."

Subsequently, a number of mathematicians discovered unusually complex behaviour in simple equations: Gaston Julia and Pierre Fatou in 1918, George Birkhoff in the 1920s, Mary Cartwright and John Littlewood in 1940s, Stephen Smale in late 1950s, and Edward Lorenz in the 1960s. Julia and Fatou laid the foundations for what became known as "fractals" (a term invented by Benoit Mandelbrot) in the 1980s, while Cartwright and Littlewood unveiled something of the complexity of oscillations. None of them attracted the attentions of the wider scientific community at the time. It was not until the mid-1970s that the different insights of mathematicians, like Siegfried Grossman, Stefan Thomae and Mitchell Feigenbaum, and theoretical biologists like Robert May and George Oster, began to come together, and it was recognised that sensitive behaviour is common and important. The word "chaos" was first used to describe it in a technical way by T.Y. Li and James Yorke in 1975.

Lorenz Atttractor - Click here to produce your own

Unpredictable weather and Jurassic Park

The story of the meteorologist Edward Lorenz's first discovery of chaotic behaviour is interesting. In 1962, Lorenz attempted to model convection in the atmosphere by computer calculations. When the ground temperature is a higher than in the upper atmosphere it causes low-altitude air to rise, which is replaced by cooler air descending. This creates a churning air flow, called convection. When conditions are just right, with no horizontal winds to break it up, the convective churning creates striking patterns of rolling cloud. But if the conditions change slightly, the convection will destroy the patterns.

Lorenz discovered some strange predictions from his computer studies of weather prediction. Like everyone else, when he began, he thought that all realistic equations of physics would produce stable predictions that would not change much if the starting conditions for the computer calculations were slightly changed. In 1929, the French mathematician Jacques Hadamard had even proposed that this type of stability as a requirement of mathematical equations which describe natural phenomena. Unstable situations could not persist and wouldn't be seen.

Lorenz found that his simple mathematical model of a weather system produced very different predictions if he slightly altered the initial weather conditions. Once, while running his program he had to stop in the middle of a run. When he wanted to resume later, he just read-off the last values taken by the three variables he was following when the computer stopped, and used them as the starting values for a new run of the program. To his surprise, the weather predictions steadily deviated from the pattern they had been following before. He couldn't understand this at first and thought that something must have gone wrong with his program. Gradually, he discovered what had happened. The computer stored lots of decimal places for the numbers it was calculating to describe the weather, but it only printed out the first few decimal places. So when Lorenz noted down the output when the computer stopped, it was as if he was approximating the solution found by the computer (recording 0.3463446510 as just 0.346 from the print out). When he restarted the program with 0.346 as the starting state the computer registered it as 0.3460000000. The difference (or "error") between the two numbers is very small (0.0003446510), but each time the computer performed an operation it would roughly double the error. Very soon the differences between the old and the new computations became huge. Lorenz realised that very small changes in the starting conditions could produce very different long-term forecasts.

Lorenz's equations

The equations that Lorenz was computing were very simple. He was following three variables, x, measuring the convective motion in the atmosphere, y measuring a temperature difference, and z measuring a variation in the temperature profile. All depended only on time t and were linked by a deceptively simple system of three equations:

where a, b, c are just constant numbers that can be altered to change the meteorological situation being studied. Lorenz published the results of his studies in the Journal of Atmospheric Science and no mathematicians noticed it.

It would be another ten years before simple systems with complicated unstable behaviour started to be studied systematically. They were given the glamourous name of chaotic systems and when people started looking for them, they found exotic chaotic behaviour all over the world of science: in the variation of animal populations, human heart-beat patterns, economic fluctuations, dripping taps, and the motions of stars in our Milky Way galaxy. Subsequently, chaos became big news, inspiring the plots of movies like Jurassic Park (where a small change in the starting conditions for a dinosaur-breeding project produces an unpredictable long-term outcome) and Tom Stoppard's play Arcadia.

Sometimes, the sensitivity of chaotic systems is called The Butterfly Effect. The term was coined by Lorenz (the title of one of his papers was "Can the flap of a butterfly's wing stir up a tornado in Texas?") to describe the extreme sensitivity of a chaotic system. It appeared that a perturbation as small as the flap of a butterfly's wing could have repercussions for the whole weather system in the future because of the amplifying feature of chaos.

You can create a complicated system like Lorenz's with a tall glass of water. Stir it very gently and release a drop of coloured ink onto the surface. You will see the flow lines traced out by the ink into curls and flow lines that loop back upon themselves and do very complicated things. If two points of ink begin close together they will soon end up far apart. You can try this with the water almost still, when the ink will spread out slowly. If you stir the water vigorously then the ink patterns will disappear without trace very quickly.

Another example can be generated in your kitchen. If you bake bread then you will have to knead a block of dough (or, if not, use plasticine). Kneading involves stretching and folding over the dough in different directions many times. If you marked two nearby points in the dough (say by putting two currants quite close to each other) then after much kneading they will have been moved around in a very complicated way and spend a lot of time far away. This is folding-and stretching operation is typical of in which many chaotic processes quickly lose memory of how they started out. It has the same effect as shuffling a pack of playing-cards.

Experimental Mathematics and PCs

The emergence of the idea of "chaos" as an important general idea, with ramifications throughout mathematics and many sciences, in the period from 1975 onwards was not an accident of sociology alone. This was the time when we began to see the first personal computers coming available. Chaotic systems are best studied by computer simulation - experimental mathematics. PCs provided, for the first time, relatively inexpensive, small, easy-to-use computers, with good graphics, that could be used by large numbers of mathematicians and scientists to explore what happened in simple systems of mathematical equations like Lorenz's. Previously, computers had been big, both in size and cost, and tended to be controlled by large well-funded research groups who focussed on the solution of huge block-busting problems.

Initially, the study of chaos needed a lot of exploration of systems of simple equations in order to build up some understanding of what all the mathematical possibilities were, and to become skilled at creating simple equations which could describe real-world situations that appeared to be chaotic. This created a new type of mathematics - experimental mathematics - in which simple equations with very complicated behaviour were watched on computer screens and investigated by altering them in different ways to find the range of interesting possibilities. From these simple explorations many far-reaching properties were discovered and then proved rigorously. Whereas in physics or chemistry you often make predictions from a theory which you test by experiment, here you explored the possibilities experimentally on a computer and then developed the theory that explained what was seen.

What is chaos?

Chaotic systems are those in which a small uncertainty in the initial state, dU(0), results in  exponential growth in uncertainty if we follow the evolution to the future (or the past) in time t;

You can regard dU(0) as the difference between two neighbouring starting states at time t=0 and dU(t) as the difference between them after a time t. In a non-chaotic system an uncertainty in the initial state would either get smaller or amplify in proportion to the time t (or in proportion to a power of t, like t²). The extreme sensitivity of chaotic systems is reflected by the exponential growth of uncertainty. This is faster than any algebraic power of the time if you wait long enough. Note that because e^t=a^mt for any constant a if we pick m = 1/lna, exponential growth is similar in growth rate to doubling (a=2) or trebling (a=3), and so on. Occasionally, you can encounter a situation that is hyperchaotic, where differences and uncertainties grow even faster than exponentially in time, as t^t.

What these studies soon confirmed was a great surprise. Mathematicians had always known that it was possible to create problems (and equations) which displayed extremely complicated, apparently patternless behaviour. But they thought that this would always require very complicated equations, perhaps dozens of interlinked equations, or equations which contained random starting conditions, or which had random changes imposed on their development every so often. While any of these situations can produce chaos, none of them are necessary:

Very simple deterministic equations (with very few variables), with definite initial conditions, and undergoing no random perturbations of any sort, could be completely unpredictable for all practical purposes.

A simple chaotic system: the circle map

Here is a simple example which shows this growing unpredictability in action. Suppose we have a law which moves a pointer around a perimeter of a circle, like a single hand on a clock face. The location of the pointer is described only by the angle that it makes with the vertical at 12 o'clock. The hand starts off at a position specified by an angle q = q₀ and then moves according to the following rule which gives the (n+1)st position-angle, q_n+1 , in terms of the nth position-angle, q_n :

When we reach 360^° we keep on going around the circle.

This rule is completely deterministic in principle. If we know the initial position, q₀ , precisely then we will also know the pointer's later position precisely after any number of applications of the rule, doubling the angle q each time we move around the circle, as we let n get bigger. Starting at 10^° , we move to 20^° , to 40^° , and so on.

Now, we want to ask what will happen if there is a little uncertainty in our initial knowledge of where the pointer is located initially, let's call it dq₀ .Then we see that this uncertainty will also be doubled each time we apply the rule. Eventually, no matter how small dq₀ is, as n grows the later uncertainty will have grown to become greater than 360^° and we will have lost track of the position of the pointer on the circle. This tells us that the system, while deterministic in principle (in an ideal world where measurements are perfect) it is not deterministic in practice.

The reason that our weather predictions are often inaccurate is not because of our poor knowledge of the laws that govern how the weather changes, it is because we have only partial knowledge (say, from weather stations every 50-100 kilometres over the land) of the state of the weather now at every point in space. Whereas lack of information about how the clock pendulum began swinging was not significant, lack of information about the current state of a weather system is.

Another worry in a system like the movement of our pointer is that if you did not get the r Another worry in a system like the movement of our pointer is that if you did not get the rule perfectly correct then there will also be huge cumulative errors produced by applying it many times. Suppose that the real law was not q_n+1 = 2q_n , but actually

qn+1=2.00001qn.

After a few applications of the rule the consequences of using 2 instead of 2.00001 from the same starting condition grows huge .

These examples show why chaotic systems are so dangerous. If you are building a mathematical model of some trend, business, or manufacturing process, and you are unaware that there is a chaotic process operating, then you could go on happily calculating away, unaware that you are generating output that quickly becomes meaningless because you had not specified the starting conditions with sufficient accuracy or not determined the rule of change involved precisely enough. Chaos is dramatic sensitivity to ignorance.

Sensitive equations

Here is an example of how an equation can change in a very fundamental way when a small change is made. Suppose that you are trying to discover if it is possible to have stable equilibrium in an ecosytem in which three things (A,B,C) can be changed. The whole search for an equilibrium reduces to the solution of a quadratic equation

Ax2+Bx+C=0.

The equilibrium is specified by a real solution for x and it is determined by the three numbers A,B,C, which you have control over. The solutions of the equation are provided by the formula

Imagine that you find that you are in a situation where B² is approximately equal to 4AC, but you can't be absolutely certain which of these two terms is going to be bigger, because of the uncertainties in measuring the values of A, B, C. But this is a very dangerous situation: if B² is just bigger than 4AC then there are two real solutions for x, if B²=4AC there is just one, whereas if B² is very slightly less than 4AC there are no real solutions for x at all. Small changes in B²-4AC produce big changes in the overall solution to the problem.

How can you predict anything about a chaotic system?

These feature might make you think it hopeless even to try to use mathematics to describe a chaotic situation. We are never going to get the mathematical equations one-hundred per cent correct for something like weather - there is too much going on - so we will always end up being wildy inaccurate in our predictions. Remarkably, this need not happen. It has been found that whole collections of equations can possess shared properties regardless of their very specific form. If our problem is really described by one of the equations in that class, then there will often be a simpler version of it which we can find that falls in the same class. It is easier to study and can be shown to have the same properties as the true equation. Mathematicians focus attention upon properties shared by almost all equations of a certain type.

Another important feature of chaotic systems is that although they become unpredictable when you try to determine the future from a particular uncertain starting value, there may be a particular statistical spread of outcomes after a long time. In the circle map, for example, there will eventually be an equal probability of finding the pointer anywhere on the circle, no matter where it started from. The more times the rule is applied to move the pointer so the better will this statement about equal likelihood of being found any where become. Finding the probability that a particular range of values will be visited is an important thing to know because most mathematical equations exhibit different degrees of sensitivity for different values of the quantities they describe.

Molecules, Snooker and quantum uncertainty

If you take a volume of moving molecules (their average energy of motion determines what we called the gas "temperature") and think of the individual molecules as little balls, the motion of any single molecule is chaotic. Each time it bounces off another molecule any uncertainty in its direction is amplified exponentially. This is something you can check for yourself by observing the collisions of marbles or snooker balls. In fact, the amplification in the angle of recoil, q, in the collision of two identical spherical molecules is well described by a rule like our q map, with

where d is the average distance between collisions and r is the radius of the spheres.

You can apply this rule to snooker balls as well as molecules. One knows from bitter experience that snooker or pool exhibits sensitive dependence on initial conditions: a slight miscue of the cue-ball produces a big miss! If the balls are bouncing around a frictionless snooker table in a perfect vacuum (otherwise they will just stop moving after one or two collisions) then we might typically have d=1 metre and r=3 cm, so our map is q_n+1 = 3q_n. The growth in recoil angle uncertainty in the trajectory of a ball as it bounces off other balls is therefore pretty dramatic. In fact, if you hit the ball as accurately as Heisenberg's quantum Uncertainty Principle allows any physical process to be determined by observation, then only about 12 collisions are needed to amplify this uncertainty up to more than 90 degrees!

Predictable things about chaos

Maxwell's intuition about unpredictable systems probably derived from his studies of the statistics of motions of gas molecules. They behave like a huge number of snooker balls bouncing off each other and the walls of their container (made of denser, more closely packed molecules). Unlike the snooker balls they won't slow down and stop. They give us a nice example of how some chaotic systems can have simple, predictable, average properties. All the motions are individually chaotic, just like the snooker balls, but we still have simple rules like Boyle's law governing the pressure, P, volume, V, and temperature, T, of the gas of molecules.

Chaotic systems can have stable, predictable, long-term, average behaviours. However, it is often very difficult to predict when they will. You usually just have to explore and discover whether they do or not.

This is also very important for computing the behaviour of chaotic systems. Many systems possess a shadowing property that ensures that computer calculations of long-term averages can be very accurate, even in the presence rounding errors and other small inaccuracies introduced by the computer's ability to store only a finite number of decimal places. These "round-off" errors have the effect of moving the solution being calculated onto another nearby solution. Many chaotic systems have the property that these nearby behaviour end up visiting all the same places as the original solution and it doesn't make any difference in the long-run that you have been shifted on to it. For example, when considering molecules moving inside a container, you would set about calculating the pressure exerted on the walls by considering a molecule travelling from one side to the other and rebounding off a wall. In practice a particular molecule might never make it across the container to hit the wall because it runs into other molecules. However, it gets replaced by another molecule that is behaving in the same way as it would have done had it continued on its way unperturbed.

Further reading:

• What is Chaos? a five-part online course for everyone. http://order.ph.utexas.edu/chaos/
• James Gleick: Chaos: Making a New Science. Copyright 1987, Viking, N.Y.
• Mitchell M. Waldrop: Complexity: the emerging science at the edge of order and chaos.
• Chaos Gallery: http://www-chaos.umd.edu/gallery.html