Seeing through expert eyes: ace decision makers may perceive distinctive worlds.
This persistence strikes the lieutenant in charge of the unit as odd. A second round of dousing proves equally ineffective. The firefighters retreat a few steps, as an uneasy feeling grips their lieutenant. No clear-cut signs of danger loom, but his intuition--he calls it his ESP, or extrasensory perception--tells him that they shouldn't be there. The lieutenant gives the order to leave.
Just as the men trudge out to the street, the living room floor caves in. They have barely missed plunging into a blazing basement.
Years later, the veteran firefighter remembers having had no inkling that the house had a basement. At the time, however, he knew that the situation violated several of his basic expectations about how fires behave.
First, the small kitchen blaze casually shrugged off water. Second, the living room was surprisingly hot considering the limited size of the fire. Third, fires that generate a lot of heat make a lot of noise, but a strange quiet pervaded the house.
Only after the incident did the unit commander realize that his team had unknowingly planted themselves above an underground inferno. Rising heat enveloped the firefighters even as the living-room floor muffled the fiery din below. With a hidden power source like that, no wonder the kitchen fire was water repellent.
In truth, the lieutenant's life-saving intuition stemmed not from ESP but from his actual fire-fighting experience. His story is one of many examples of decision-making expertise collected over the past 20 years by psychologist Gary Klein.
The term "expert" currently conjures up images of overeducated bumblers and cynical data manipulators who sell convenient testimony to one side or the other in court cases. According to modern folklore, as stated in the June 5 Science, "For every expert, there is an equal and opposite expert."
In contrast, Klein is helping to develop a research perspective--known as naturalistic decision making, or NDM--for unraveling how people become bona fide experts in performing complex, real-life tasks.
"In many dynamic, uncertain, and fast, paced environments, there is no single right way to make decisions," Klein says. "Experts learn to perceive things that are invisible to novices, such as the characteristics of a typical situation. They make high-quality decisions under extreme time pressure. When difficulties arise, experts find opportunities for improvising solutions."
For more than 30 years, investigators interested in decision-making have largely focused on the performance of laboratory tasks or word problems with single, preset solutions. The results portray individuals as relying on simple thinking guidelines that often distort judgment, such as a tendency to see what one expects to see. Experts may harbor a sense of infallibility that renders them especially vulnerable to such cognitive biases, according to this school of thought.
A small contingent of researchers, based mostly in Europe, have taken a different approach. They have studied people engaged in complex jobs and viewed expertise as a product of extended experience with a particular task, such as operating a nuclear power plant's control console. With the information they have gathered, they have improved job training programs and made technology more user friendly.
Building on the European approach, an international group of researchers is now exploring judgments in real-life, sometimes life-or-death, endeavors. These investigators usually take pains to distinguish experts from less skilled practitioners. The field of naturalistic decision making has attracted many interested parties, including psychologists, physicians, mechanical and software engineers, air traffic controllers, and members of all branches of the U.S. armed forces.
The Fourth Conference on Naturalistic Decision Making, held in May in Warrenton, Va., featured the latest research in this growing area. It was hosted by Klein Associates of Fairborn, Ohio, a consulting company that Klein founded in 1978. He describes his views on decision-making and expertise in his book Sources of Power: How People Make Decisions (1998, MIT Press).
Klein and many others conducting research on naturalistic decision making take inspiration from the idea, advanced by Hubert L. Dreyfus and Stuart E. Dreyfus in their book Mind Over Machine: The Power of Human Intuitive Expertise in the Era of the Computer (1986, Free Press), that people who repeatedly confront a particular task move in stages from the rank of novice to that of expert. Rather than sifting through a vast stockpile of rules, facts, and wisdom, experts use their experience to recognize distinctive situations that, in turn, evoke helpful tactics and actions, according to the Dreyfuses, both of the University of California, Berkeley.
Klein heard echoes of this notion as he and his coworkers inter viewed firefighters, Air Force pilots, nurses, and other workers in efforts to improve job training and performance. He calls his own spin on the Dreyfus' idea the recognition-primed decision model.
Klein's model holds that experts pressed for time while facing a task loaded with uncertainty first assess whether the situation strikes them as familiar. Rather than comparing in detail the pros and cons of different approaches, they quickly imagine how one or a few possible courses of action in such situations will play out. Experts take the first workable option they can find; it may not be the best of all possible choices, but it often yields remarkably good results.
A limited search of relevant clues to fairly difficult problems aids decision making in general, according to related findings (SN: 7/13/96, p. 24).
Studies of chess players support the notion that experts rapidly discern helpful patterns rather than running down lists of rules and procedures. In one of Klein's studies, he considered the moves that three chess masters made at critical junctures, as defined by consulting chess grand masters. Whether the masters were competing against each other in regulation games or under blitz conditions that allowed an average of only 6 seconds per move, their moves were consistently ranked as high quality.
In contrast, the performance of three relatively weak chess players plummeted in blitz games, apparently because of an inability to perceive quickly the implications of the positions of key playing pieces.
A second investigation led by Klein required chess players with high or medium skill ratings to discuss aloud every move they considered upon seeing four different chess positions. At both playing levels, the majority of the first moves that occurred to the players received good ratings from a consulting grand master. Most of the other potential moves, however, were not worth taking, according to the master. Although high-skill players made the best moves, all players agreed with the master that the first moves they thought of were usually their best moves.
In the less structured and far riskier arena of military warfare, recognition-primed decisions represent one of several weapons wielded by effective troop commanders, Klein argues. In a Marine Corps-sponsored study, he and U.S. Marine Corps Major John E Schmitt spent a year interviewing regimental commanders and observing their responses to computer simulations of battle situations. Based on characteristics determined by historical reviews of military commanders, they deemed a small minority of their subjects to be expert.
The expert commanders share no single recipe for concocting battlefield strategies, but they all grapple well with warfare's shifting, uncertain conditions, according to Schmitt and Klein. Tactical moves evolve out of simple plans that can be rapidly altered, either out of necessity or as opportunities arise.
When confronting familiar situations, the leaders who are rated as superior explain to subordinates in clear language what they see and what they want to do. Subordinates' input then can be solicited and pooled to determine quickly whether events are proceeding as intended in the heat of battle.
Effective commanders also use available information to consider the adversary's perspective and thus anticipate enemy actions. As in a chess match, recognition of familiar patterns in an opponent's tactics aids planning far more than trying to collect as much information as possible.
Finally, expert commanders almost always take actions to restrict the enemy's options and reduce the amount of uncertainty in the situation. New tactical opportunities emerge as the opposing forces react to the challenge.
The less effective commanders, in contrast, find it difficult to formulate decisions given the intense anxiety evoked by the uncertainties of warfare.
Despite the tremendous practical power attributed to genuine expertise, researchers have yet to devise a unified theory of decision making, says Kim Vicente, a mechanical engineer at The University of Toronto.
As a first step in that direction, Vicente nominates four factors as major orchestrators of real-world decisions. First, the environments in which people operate contain a certain level of orderliness or structure relevant to what the people want to accomplish. For instance, certain physical properties of the Earth and its atmosphere have a large influence on weather patterns.
Second, people perceive cues that provide insights into an environment's structure, such as the jet stream patterns and cloud thickness measures often consulted by weather forecasters.
Third, individuals and groups have a range of capabilities that influence how well they can achieve their goals. For example, a military unit's success rests on the effectiveness of the commander and the coordination of roles for subordinates.
Finally, decision makers differ in the extent to which they modulate their plans and behaviors as events in the environment unfold.
As Vicente's framework implies, even proficient judges have to work within the limits of predictability for the task at hand, asserts psychologist Thomas R. Stewart of the State University of New York (SUNY) at Albany. A study directed by Stewart, published in the March 1997 Organizational Behavior and Human Decision Processes, serves as a case in point.
Four SUNY atmospheric sciences students demonstrated considerable accuracy and agreement in next-day predictions of temperature and rainfall over 178 days, outperforming a computerized weather prediction model, Stewart and his colleagues reported. Both the students and the computer model had access to all information used by National Weather Service forecasters, including satellite images and radar.
On the other hand, meteorologists often make mistakes in predicting hail or other types of severe weather, which were not considered in the student experiment, Stewart says. These incidents occur in more unpredictable ways that cannot be easily derived from available weather data.
A review of 18 studies of various types of expert judgment, presented last year by Stewart and a colleague at a scientific meeting, indicates that experienced physicians, teachers, psychologists, and others try to make the most out of cues that often provide limited insight into what they Want to predict. For example, psychologists attempting to predict patients' mental diagnoses based on their scores on standard psychological tests do rather poorly. The task is inherently as unpredictable as trying to forecast a hailstorm, Stewart argues.
"In other words, the major problem with expert judgment is with the environment, not with the expert," he contends. "The experts are doing about as well as they can, given the information they have to work with."
Sometimes it's possible to tailor the information that experts have to work with and thus to boost their performance. People can represent the same information in different ways to solve a problem, assert Ulrich Hoffrage and Gerd Gigerenzer, both psychologists at the Max Planck Institute for Human Development in Berlin. They asked a group of 48 experienced physicians to estimate the probability of someone having a disease for which he or she has received a positive test result. The doctors gave more accurate estimates when the relevant statistical information was presented to them in one form rather than another, Hoffrage and Gigerenzer report in the May Academic Medicine.
Take the case of a 40-year-old woman who has a positive mammogram. If told that 10 out of 1,000 women have breast cancer at age 40, eight of those 10 will have a positive mammogram, and 99 of the remaining 990 women will also test positive, about half of the physicians calculated correctly that eight out of 107 women testing positive do have breast cancer.
Nearly all physicians, however, overestimated a woman's chances of actually having breast cancer--usually by a long shot--if told that the probability of having breast cancer among 40-year-old women is 1 percent and the probability of a woman with breast cancer having a positive mammography is 80 percent, compared with 10 percent for a woman without breast cancer.
While user-friendly information can lighten a mental load, it cannot banish uncertainty from decisions in medicine and other fields. Experts inevitably disagree, since they face complex problems that have no single correct answer, says psychologist James Shanteau of Kansas State University in Manhattan. Like Klein, he sees the expert's role as hinging on the recognition of patterns and consistencies that clarify options in complex situations.
Experts also diverge because' improvisation is a part of their repertoire, adds psychologist Karl E. Weick of the University of Michigan in Ann Arbor. Accomplished improvisers appear to make provisional sense of a situation--without actually reaching a decision--by launching experience-based actions that in turn trigger creative revisions. "When people improvise, they act in order to think," Weick says.
For expert physicians, this means that they may choose to abandon the traditional sequence of first noting symptoms, then generating a diagnosis, and finally administering a treatment. Instead, they start with a plausible treatment as a tentative way to control a patient's puzzling symptoms; during the treatment, the physicians may notice other symptoms that help in revising the diagnosis. A more appropriate course of treatment is eventually determined.
Researchers examining naturalistic decision making will need a similar feel for crafting informed innovations if they hope to become experts on expertise, according to Klein. Considering their accomplishments to date, that goal does not seem unreasonable, he proposes.
"The accumulation of experience doesn't weigh people down," Klein says. "It lightens them up."
|Printer friendly Cite/link Email Feedback|
|Title Annotation:||intuition and experience in decision-making|
|Article Type:||Cover Story|
|Date:||Jul 18, 1998|
|Previous Article:||Leptin augers heart attack, diabetes?|
|Next Article:||What aircraft leave behind.|