Wizard of Oz: bringing drama to virtual reality.

When Dorothy steps out of her drab, cyclone-tossed farmhouse into the glowing Technicolor landscape of Munchkinland, she brings with her not only her dog Toto, but also the film's entranced audience. Captivated by a compelling story and vivid characters, viewers of "The Wizard of Oz" willingly surrender themselves to the fantastic world through which Dorothy and her companions travel.

Now imagine yourself participating in the adventure - skipping down the yellow brick road with Dorothy, conversing with the Scarecrow, sparring with the Cowardly Lion, confronting the Wicked Witch of the West in her castle - and influencing the action.

Such is the promise of new forms of entertainment that rely on computers to create interactive, simulated worlds.

With the technology for generating these microworlds still at a rudimentary stage, most researchers have concentrated on such technical matters as making the required paraphernalia-goggles, instrumented gloves, video screens less intrusive and on integrating the hardware so it functions seamlessly. These efforts have produced systems that allow users to play interactive games, "stroll" through virtual buildings, and wage war across simulated battlefields (SN: 1/4/92, p. 8).

But that's not enough for computer scientist Joseph L. Bates, who heads the Oz project at Carnegie Mellon University in Pittsburgh. Bates wants to learn how to build convincing characters to populate these computer-mediated worlds. He wants to develop software that authors can use to fashion rich, dramatic frameworks for guiding the activities of both user and fictitious character.

"To me, the interesting part is not the interface question of how you present something in three dimensions and so on," Bates says. "Well-written books do a wonderful job of projecting you into a fantasy world. It's really a question 01 the quality of the characters and the quality and style of the story that goes into the presentation."

In Bates' view, the denizens of artificial worlds must themselves be active. "They have to do things, and in order to do things, they need to know what to do:' he contends. "We have to build in enough of a mind to convince a user who's interacting with the character that it's alive and complex."

To explore these issues, Bates has gathered together a research group comprising faculty and students in computer science, drama, and English. "It's a great mixing of the arts and science," he notes. "It involves psychology, it involves art, it involves computing."

At the Oz project's core lies the notion of applying and integrating what researchers in artificial intelligence (AI) have learned about developing software that can represent emotions, formulate goals, direct behavior, and understand language. "We have here microworlds that are artistically interesting:' Bates says. "This is an area in which some of the goals of AI can be achieved without having the complications of the real world."

Lyotard, a temperamental house cat, lives in a computer. The only readily apence of its existence is the printed text that appears on a computer screen.

Bates, graduate student A. Bryan Loyall, and their co-workers created this simulated cat, who inhabits a virtual apartment, as part of an experiment in "interactive fiction." They sought to construct a creature that reacts convincingly enough to persuade any person invading its territory that it's a real cat.

Cat owners in the research group furnished a catalog of feline behavior on which to base the simulation. Software linked these characteristic behaviors to representations of such emotions as anger, fear, hope, sadness, dislike, happiness, and gratitude.

The story itself concerns the cat's behavior in the absence of its new owner when a visitor comes to the apartment. What happens depends on how the cat is feeling and on what you, as the visitor, key into the computer. In response to these typed stimuli, Lyotard senses, thinks, then acts. The result appears as text on the computer screen.

Despite its complete dependence on text, this simulation can be remarkably engrossing. Users report spending lengthy periods trying to get Lyotard to come out of hiding or to nibble on a snack.

This success emphasizes that simulated beings don't have to be especially smart, and they don't have to reside in a realistic, three-dimensional environment for a user to suspend disbelief. "Just as in a good film, it's usually not the scenery although it's nice to have good scenery - but the character or story that's at the film's core," Bates notes.

Nor do characters have to be overly complex. Indeed, a simulated creature that keeps quiet most of the time can appear quite knowing. It takes advantage of a human tendency to attribute subtlety, understanding, and emotion to such subdued behavior - so long as the creature doesn't actively destroy this illusion by doing something patently dumb or unrealistic.

Good novelists have numerous literary devices at their disposal for carrying readers through the twists and turns of a plot to a thrilling and satisfying conclusion. But what can the author of a free-running dramatic simulation do in the much more delicate situation of shaping the experience of a user apparently free to interact with the fictitious characters?

"This is a kind of computational mixing of free will and destiny," Bates says. "We want you, the user, to be free to act as you wish, yet the artist will often want to impose a kind of destiny So it has to be imposed sufficiently gently that you don't feel manipulated."

To learn what kinds of instructions it would take to channel an interactive experience, Bates and his collaborators in the drama department have gone onstage. Their "live" simulations feature actors playing both the fictitious characters and the user. This setup also requires the presence of an offstage director.

The story begins with the actor playing the user coming onstage to interact with the characters already present. The director follows the action, thinks about what's happening, ponders options, and makes suggestions to the characters but not the user - via radio to keep the scenario on track.

"The director has long-term, global control of the experience, while the actors interact locally," Bates says. "Our goal is to learn what the director needs to think about, what the director communicates to the actors, what the actors need to understand, and how that corresponds in the computer to the components concerned with story and the planning of possible futures' as an interaction proceeds."

The issue boils down to determining the extent to which a director-whether a human being or a computer program can influence activity in a microworld without attracting the user's attention.

The researchers have learned that in an interactive setting, people are more likely to resent the obtrusive use of music, lighting, or props to alter behavior than the actions or words of characters. Thus, a director can impose direction most effectively and unobtrusively by calling for the characters to make slight adjustments in their behavior to get a desired result.

"In retrospect, this particular method of wielding power in subtle ways may be already well known in society," Bates wryly notes.

At the same time, people engaged in an interactive experience are remarkably forgiving of the foibles they detect in fellow humans. They easily accept odd behavior in actors. "You don't feel manipulated:' Bates says. "You readily make up reasons for almost any strange thing that an actor does."

Last July, at a meeting of the

American Association for Artificial Intelligence held in San Jose, Calif., Bates organized what he described as the first AI-based arts exhibition. "AI is particularly crucial to future interactive art because the works of art must themselves include artistic knowledge and abilities," he wrote in the exhibitions program. "They need these to respond in artistically appropriate ways as they interact with the viewer, who is now a participant, without the original artist being present."

One exhibit featured music composed with uncanny fidelity to Mozart's style by a computer program called Experiments in Musical Intelligence, developed by David Cope, a music professor at the University of California, Santa Cruz. Another, representing the work of doctoral student Janet Cahn of the Massachusetts Institute of Technology's Media Laboratory, rang with the sounds of synthesized speech -- carefully tuned to express a range of emotions.

The piece presented by Bates and his collaborators put four animated characters in a three-dimensional landscape on a two-dimensional computer screen. Three of the rotund, big-eyed, candy-smooth characters were controlled by a personality embodied in software that integrates reactivity, goal-directed behavior, and emotion. The other was controlled by any visitor to the display

Sonar sensors detected people approaching the monitor. In response, the animated characters could show curiosity, fear, or some other emotion and behave correspondingly, A visitor could then use a pointing device - a mouse - to move one character about and interact with the others.

"Our goal was to build creatures believable enough that people would internally have to make a decision whether to treat them as objects or as living things," Bates says.

To emphasize this point, the researchers placed near the monitor clear glass vases filled with jelly beans colored to match the creatures on the screen. "People had to decide whether to eat those 'creatures' or to begin treating them as emotional beings worthy of human respect:' Bates says. "Our title, 'Edge of Intention: suggested that we are approaching the point when simulated agents will have us wondering about what ethical responsibilities we may have to these creatures."

Just as novels are much more than print and paper, interactive media are much more than video screens and computers. To make the results sufficiently inviting and dramatic requires focusing on what it takes to create rich, story-like environments and engaging characters.

But the fact that the reader or viewer is also a participant complicates the situation. "It's not as simple as the technology of film or writing or radio," Bates says. "You're really building an artist as well as a piece of art."

Bates envisions the day when authors will have a conceptual framework and a set of "languages"- packaged as computer programs - for assembling the elements of story, behavior, and emotion into compelling, simulated microworlds populated by fascinating creatures.

"It's a large project," Bates admits. "We're trying to bring together a lot of different technologies. It's a mixing of many different structures and ways of thinking."

For the moment, participants in the Oz project are concentrating on giving their simulated creatures the ability to speak and to understand speech. They also want to add the kind of long-term dramatic control of a simulated world that they tested on the stage.

"My goal is to build the fundamental technology and get it into the hands of individual artists or small groups of artists, and then let them drive the process," Bates says.

Armed with such tools, a master storyteller of the future may someday bring a startling new dimension to the cinematic wizardry that effortlessly sweeps Dorothy and her audience away to Oz.

That's entertainment!