Basins of froth; visualizing the "chaos" surrounding chaos.

There are more things in heaven and earth, Horatio, Than are dreamt of in your philosophy.

At first, James C. Alexander didn't know what to make of the stark image - just two crossing lines - projected on the screen. The emergence of this "X" from a set of simple equations being studied by a physicist didn't seem to fit with what Alexander knew of the mathematical behavior of so-called dynamical systems.

"It was a complete mystery," Alexander says of his first glimpse at a dynamics meeting two years ago of a remarkable pathology that appears to afflict certain types of equations.

"In fact, while everybody else was having a nice dinner that night, I was scribbling on a place mat trying to figure out what was going on," he recalls.

Alexander's pursuit of this aberrant behavior did more than confirm the presence of the unlikely X. It eventually unveiled a bizarre mathematical realm even stranger, and in some sense wilder and more unpredictable, than that found in dynamical systems now commonly described as chaotic.

"If you had told me a year ago that such a phenomenon could exist in a robust sense in dynamical systems, I would have said it can't happen," says mathematician James A. Yorke, one of Alexander's colleagues at the University of Maryland in College Park. "Yet here it is."

This discovery adds yet another surprising element to the growing stock of exotic behavior arising out of the manipulation of simple mathematical expressions. "Clearly, in a philosophical sense, Nature isn't done throwing curves at us," Alexander remarks.

Dynamics deals with change, and mathematicians interested in dynamical systems study how a system, defined by a set of equations, shifts from one state to another. For example, given the coordinates of a starting point, a set of equations (termed a "mapping") supplies a way of computing a particular system's new coordinates (or state) one unit of time later.

Applying the same equations to the newly computed coordinates generates the system's state after a second unit of time has passed, and so on. Such an iterative procedure generates the coordinates of a chain of points, called the "orbit" or "trajectory," corresponding to the original point.

Mathematicians are particularly interested in what happens to these orbits in various dynamical systems. In some cases, for instance, certain collections of starting points lead to the same end points. or to a particular group of end points. Such "final" states - whether a single point or an array of points - are known as attractors, and the area covered by starting points that eventually arrive at an attractor is called a basin.

For certain equations, even slight changes in the starting point lead to radically different sequences of orbit points. At the same time, it becomes virtually impossible to predict several steps ahead of time precisely where a particular trajectory will go. This sensitive dependence on initial conditions stands as a hallmark of chaos.

Despite this sensitivity, however, chaotic trajectories in a given dynamical system still generally end up on an attractor that has a particular geometry -- albeit one that can look extremely convoluted and complicated. But such an attractor normally doesn't contain a crisp, unambiguous X.

"You don't expect crossings or sharp corners," Alexander says.

By taking a close look at the physics equations that originally captured his attention, Alexander came to realize that the corresponding dynamical system apparently has three attractors. "The attractors in this case are incredibly simple," Alexander says. "They're just line segments."

These line segments intersect to form the outline of an equilateral triangle. The physicist's X was merely one of the three places where the line segments cross. "He was looking at only a small part of what was going on," Alexander asserts.

Alexander found that any starting point already on one of these lines will follow a trajectory that skips erratically back and forth along the line without ever hopping off. This unpredictable behavior furnishes evidence that the lines themselves are chaotic attractors.

The surprise comes in the behavior of starting points chosen from areas near the lines or inside the triangle. Whereas one starting point follows a trajectory leading to one of the line-segment attractors, another starting point only a tiny distance away may end up on a different attractor. There's no way of predicting on which attractor a given starting point will land.

"Normally, if you start at some point and then start at a slightly different point, you generally expect to come down on the same attractor," Alexander says. In this case, "if a point goes to one attractor, then arbitrarily close to it there are points that go to another attractor."

In other words, the system's extreme sensitivity to initial conditions determines not only where on an attractor a given point will land (as in chaos), but also on which attractor it will fall. Hence, instead of having a single basin of attraction, this dynamical system has three thoroughly intermingled basins.

Using modern computer graphics to portray this newly unveiled, erratic behavior reveals an astonishingly rich landscape of filigreed features. The resulting images, with three different colors used to represent starting points that end up on each of the three attractors, clearly show the intricacy of the meshed basins belonging to the attractors.

Such images suggest the idea of a "riddled" basin. "Each basin is just shot full of holes," Alexander says. "The idea is that no matter where you are, if you step infinitesimally to one side, you could fall into one of the holes, meaning you could end up at a completely different attractor."

"These things are like a foam of soap bubbles," he notes. "It's very hard to tell whether you're on the soap film or in one of the holes."

But pictures by themselves aren't enough to characterize a dynamical system. "We've been working largely from pictures, but this is such a sensitive phenomenon that it would be really nice to have it pinned down mathematically," Alexander says.

Alexander and Yorke, together with Maryland's Zhiping You and Ittai Kan of George Mason University in Fairfax, Va., have started providing such a mathematical framework. Their preliminary results appear in a paper to be published in the INTERNATIONAL JOURNAL OF BIFURCATION AND CHAOS.

How common and how important is the phenomenon of intermingled, riddled basins? No one knows yet, but Yorke has identified a second set of equations, sometimes used in mathematical biology to describe fluctations in the populations of two competing species, that displays similar sensitivities.

"It's not clear whether these are isolated examples or whether they really occur commonly," says mathematician John W. Milnor of the State University of New York at Stony Brook. "I think there's much work to be done in finding out just how prevalent this kind of behavior is."

Indeed, many mysteries remain.

For example, computed images show that the probability that a starting point will end up on a given attractor varies, depending on the site of departure. In other words, although one can't predict on which attractor a given starting point will land, one apparently has a higher probability of reaching a particular attractor by selecting starting points in certain areas.

"We want to find out exactly how the probabilities work," Alexander says. "Can you quantify them in any way?"

What's remarkable about this work is that nothing about the equations themselves, derived from applications in physics and biology, has changed. This strange behavior has always been there, but no one had previously thought to look for it.

Now mathematicians suddenly have a new realm to explore in what had seemed a thoroughly familiar, commonplace world. And researchers can begin to look for evidence of equivalent wild effects in physical or biological systems.

"The phenomenon of intermingled basins is not, I'm sure, as widespread as chaos," Yorke says. "Nonetheless, it indicates that we may think we know what's happening, but we have blind spots. How many strange things are there out there? How much more has everybody missed?"

"It's not at all clear what the ramifications and nuances will be," Alexander adds. "There's a lot to do and lots of things to look at. It's like the beginning of a walk into a forest."