Mastering the Game
Roger Lewin and Birute Regine

New Scientist, 26 September 1998, pp 38-42

John Holland spends his summers in a house in the woods near the shore of Lake Michigan, a stone's throw from the Canadian border. He describes his hideaway as "a place of peace, long horizons, and the ever changing face of nature". This seems the perfect setting for a man who draws so much inspiration from nature: he has been called Mr Genetic Algorithm for his invention of computer programs that change in ways that echoes natural evolution. "Biology is such a rich source of ideas," he says. "My forte is to take those questions, chew off a lot of stuff, and finally form what for me is usually a computer model that gives a cartoon sketch of the question." These days, the models he's thinking about are of complex systems, from economies to ecosystems, which behave in ways that stump even the most progressive mathematicians. Holland, then, has shed his old sobriquet and donned another: Mr Emergence.

Emergence is ubiquitous in nature, from simple interactions between ants that lead to complex behaviours at the level of a colony, to the endless variety of antibodies that the human immune system can generate from a small number of genes. In all cases, the behaviour of the whole is so much more complex than the behaviour of the parts. Holland describes the phenomenon as "much coming from little", which creates an aura of mystery and paradox. "Our present understanding of emergence is not much better than the child saying that Jack Frost explains the wondrous colours of autumn," acknowledges Holland. "That kind of explanation might stir our imagination, but ultimately it is unsatisfying." Although some who study complexity suggest that this mystery puts emergence beyond the reach of scientific enquiry, Holland insists that "with the right kind of science, one day we will come to understand it." He admits, however, that "we are a long way from that point as yet."

What is known about emergence is that it appears in systems in which a few simple rules govern the interaction of the component parts. Take a cellular automaton, for instance. This "computer" consists of a grid of squares, each of which changes its state--its colour, say--at regular intervals according to rules that depend on the states of the squares around it. These rules can be extremely simple, yet the behaviour of the entire device can be very complex. Holland wants to find a path for understanding the dynamics of such interactions in the real as well as artificial worlds, to discover how "much can come from little". Before we can understand emergence, there first needs to be a theory to explain it. But even this is some distance away, Holland notes. It isn't even clear yet which features are essential to emergent systems and which are incidental. And there is no mathematics capable of modelling it. This is the terrain that Holland is negotiating today--identifying those essential features and building an intellectual framework in which to consider emergent systems of whatever type.

Even to reach his present position, Holland has travelled a great distance. Though he didn't realise it at the time, his journey began many, many years ago. It all started with games.

"My father's family were great card players," Holland says. "I remember when I was very little, seeing the grownups sitting around playing pinochle in my grandparents' house. Boy, did I want to join in, but I couldn't because this `wasn't for kids'." But back at his own home he was allowed to join in. "One of my earliest memories is of playing checkers with my mother, when I was about four years old. It was fun. And, as I had a younger sister, I enjoyed having my mother's undivided attention. Sibling rivalry, you know." A few years later, his sister was allowed to join Holland in the game circle, the two siblings playing bridge with their parents. Games became an obsession for Holland, not so much for the playing, or the winning or losing, but more the process. "The minute I understood that games were based on a few simple rules that are agreed on by everyone, and that endless consequences flowed, different every time, I was delighted, fascinated," Holland says. "Pretty soon I was making up games of my own, making whole new artificial worlds, and seeing what happens. It was like playing God." Holland has been making up games ever since, only these days they are more appropriately described as scientific models, abstract cartoons that capture the essence of some part of nature cast as computer algorithms.

Holland also discovered early on that he had a facility with mathematics, much encouraged by his parents. He went to the Massachusetts Institute of Technology in 1946, to pursue it in more depth, but he was much more interested in exploring how electronic computers might be used to capture thought processes. He continued this odyssey when he joined IBM, where he became entranced with a new theory of how the brain learns and adapts, the brainchild of Canadian psychologist Donald Hebb. Hebb's was a connectionist theory, in which the connections–synapses–in the huge and apparently random network of nerve cells in the brain constantly change, in response to the organism’s experience. Holland spent much of his time as one of a few pioneers trying to model this notion, a pursuit that came to be known as neural network simulation, which much later became a powerful tool for machine learning. The experience of this early exploration influenced much of his thinking for the next three decades.

Also influential was the work of Arthur Samuel, whom IBM hired to help develop reliable vacuum tubes, which were the guts of electronic computers in those antedeluvium days. But, like Holland, Samuel spent most of his time working on what really interested him--he wrote a program that could not only play checkers, or draughts, but could also learn and improve. He succeeded, and the checkersplayer soon surpassed its inventor in skill. By 1967, it was playing at world championship level and is still unbeaten today, one of the great achievements in this brand of model building.

Simple rules, a universe of possibilities, and with the capacity to adapt, the checkersplayer program was a source of wonder to Holland. He soon realised that programming computers for IBM was not to be his future, and so he entered a doctoral program in mathematics at the University of Michigan, Ann Arbor, where he has been ever since.

Two intellectual ships sailed across Holland's bow during his doctoral program, the result of which was to change his course, literally and figuratively. The first was his reading of Herman Hesse's novel Magister Ludi, The Master of the Game. In it people are in a complex game that involved weaving endless novel themes by moving glass beads on an abacus. "Of all the things that have influenced the way I do things, Hesse's novel ranks first," Holland says. "It's the essence of what I mean by inventing things." The second was another book, The Genetical Theory of Natural Selection by R.A. Fisher, the foundation of modern population genetics. "I've always loved reading about biology," Holland explains. "Biology is a great teacher." In this case, it led Holland to play God again, to develop his genetic algorithms, which harness evolution to find optimal solutions to problems. Long regarded by many of his peers as distinctly weird and unlikely to be in the least bit interesting, let alone useful, genetic algorithms are used today as powerful tools for solving complex problems of many kinds, including in business.

Emergence as a general phenomenon to be explained is relatively new. Holland says that for a long time he understood that his genetic algorithms were what complexity theorists call complex adaptive systems. Namely, systems in which what emerges is not only greater than the sum of the parts but also change–adapt–as a result of their experience. And he says he knew that the unfolding that flowed from their simple rules could be described as emergence, which is the essence of "greater than the sum of its parts." But it wasn't until the late 1980s, when he started visiting the Santa Fe Institute in New Mexico, the Mecca of complexity theory, that he came to see emergence as a body of knowledge, a ubiquitous phenomenon with common roots.

"I started having conversations with lots of people, in physics, in economics, in biology," he says, "and I came to see that the rules of the game were much the same, no matter what field you look at." He soon became convinced that emergence was Big, in a way that people at Santa Fe had not fully formulated, so much so that he wrote the following in his recent book: "We will not understand life and living organisms until we understand emergence."

As for what emergence actually is, Holland is necessarily vague. "None of us has a solid grasp of emergence, far less a full definition," Holland concedes. "Emergence is multifaceted, and if you try to be too precise, you will lose what you're after."

Emergence can be described as a holistic phenomenon, because the whole is more than the sum of the parts. But Holland balks at loose use of the word, because it is often enveloped in a veil of impenetrability. "I hear people talk about holism in a way that avoids investigation," Holland observes. "I hear people say that because, for instance, the business environment is a complex system, is holistic, you can't plan. It's a bit like saying I can't plan in playing chess, because there are so many ramifications, and I don't know what my opponent will do, and so on. All of that is true, but I'm not going to do well at chess if I don't plan, am I?"

Holland's approach to pinning down emergence is through his models. "I'm talking about models in the sense that Newton's laws of motion form a model of the universe," he says. "These laws have a lot in common with a game. The equations describe `rules' of the game in which `moves' can be made with the help of the tools of mathematics." And just like the rules of a game, physical laws do not encompass everything that can flow from them. For instance, "Newton could not have guessed that his equations would reveal the gravity-assisted boost that takes space probes to outer planets," Holland notes.

"The same is true of the five axioms of Euclidean geometry, which are still providing surprises after centuries of study," he says. And with chess. "The rules of the game have been known for a millennium, but new and more powerful combinations are constantly being discovered," Holland says. "This should make us realise how little we probably know about the Universe, for which the rules are unknown and is vastly more complex than chess."

A mathematical model of emergence would be like manna from heaven. It would certainly be a miracle, because most established mathematics describes linear systems in which, for example, the doubling of an input doubles the output. Emergent systems are anything but linear--the state of each square in a cellular automaton, for example, is not decided by a simple sum or multiplication. "Ninety five per cent of the mathematics I know is by the board for my kind of work," Holland says.

One device that can handle large numbers of nonlinear interconnections is the computer--Holland's favoured tool. He views models of emergence in a setting that mimics an idea that goes back to classical Greece. "The Greeks argued that all machines can be constructed by combining six elementary mechanisms, namely the lever, the screw, the inclined plane, the wedge, the wheel, and the pulley," Holland explains. "In an intuitive way, this leads me to look at emergence in terms of elementary mechanisms and procedures for combining them. It's the interaction among these mechanisms, or building blocks, that generates emergence."

Holland's models, which he calls constrained emergent procedures (CGPs), are made up of a number of virtual mechanisms, each of which computes an output that depends on the state it is in and the inputs it receives. This may seem an unremarkable device, but as Holland says, it's the interactions that count. He begins to plug the output of one mechanism into the inputs of others, he creates feedback loops or attenuates the signals travelling between mechanisms. With such techniques he can build first a cellular automaton, then a neural network and even Samuel's checkers player. By feeding one type of mechanism into another he can create CGPs that change their own structure, much as the nature of stock markets change their nature as their members make their trades.

With his models, Holland can also explore the idea that one emergent system can itself become a mechanism--or building block--for a "higher" system, and so on. For example, a pond is a complex adaptive system made up of the organisms--the building blocks--living in it; the pond is then a building block for the meadow ecosystem, a complex adaptive system at a higher level; the meadow ecosystem is then a building block in the regional ecosystem, a complex adaptive system at a still higher level. And so on, right up to the global ecosystem. "It's building blocks all the way down," Holland quips, "and I'd like to get closer to the notion of what that means."

But getting closer isn't going to be easy. Once again, mathematics--or the lack of it--gets in the way. "I need to be able to work with the notion of layers of models, and mathematicians haven't thought much about that," says Holland. "The branch of mathematics that is relevant here is combinatorics, and this is one of the least developed parts of mathematics."

By giving access to the processes of emergence, Holland hopes that his models will help to identify what he calls the "lever points" in complex systems, processes that when perturbed have a great influence in the patterns that emerge. "You then have the potential to influence emergence," he suggests, "such as enhancing innovation or producing a more creative and adaptive business."

Arcane in the extreme to mere mortals, Holland hopes that this new class of models will lead him to a new form of mathematics that will penetrate the mysteries of emergence. In any case, he expects--has faith anyway--that this approach will bring into view the mechanisms of emergence, and will create "a context so stark that nothing lies hidden in the complexity, nor is there any room for mysterious, unexplained activity."

In contrast with most people who study complexity, Holland sees his work as a continuation of the traditional reductionist approach in science. "Reductionism has been tremendously powerful, and it's amazing how much has been achieved that way," Holland notes. "You take a system, study the parts, and you can understand a lot about the system." But to some people this approach sounds coldly analytical, lacking in wonder. "Not for me," he responds. "I find it more exciting, not less, when I understand how something works. There's a tendency for people to talk about emergence as surprise. ‘Emergence is what surprises you,’ that kind of thing. So if you come to understand the system, how and what it generates, then it is no longer emergence, by definition. To me that's wrong. It is true, though, that surprise can be a signal of emergence, but it doesn't define emergence. For me, there's a sense of wonder, of awe, about emergence, and that won't disappear when I understand it."