The influence of experience with a set of simulated patients on diagnosis of simulated patients not previously diagnosed.

Key Words: Accuracy, Categorization, Clinical judgment, Diagnosis.

According to Rose,[1-3] Sahrmann,[4] and the recently published Guide to Physical Therapist Practice,[5] a clinical diagnosis that is relevant to the issues commonly addressed by physical therapists is essential for making optimal patient-management decisions. Once diagnostic categories are developed, however, presumably they must be used correctly to help ensure optimal results. Unfortunately, studies of various health care professionals (eg, clinical psychologists and physicians) indicate that diagnostic judgments are not always accurate.[6-8] In some cases, diagnostic errors may be due to deficits in the clinician's knowledge. In other cases, diagnostic errors may be the result of faulty acquisition of information or faulty information processing. The focus of this article is on one type of information-processing error referred to as a similarity-based response error.

The diagnostic process has been studied from numerous perspectives.[9-14] One of the most recent approaches is to slew clinical diagnosis as a categorization task.[15] The number of studies related directly to clinical diagnosis is limited. The literature on categorization, however, offers a wealth of theories, experimental approaches, and data that can be applied if clinical diagnosis is viewed as a categorization task. For example, based on recent reports in the categorization literature, there is evidence that sometimes people rely on similarity to old examples when they are asked to categorize new examples.[16] This occurs even though relying on similarity to old examples may result in responses that conflict with a previously learned, valid rule for correct categorization.[16] The implication of this finding is that physical therapists who make diagnoses for new patients based solely on the patients' similarity to previously seen patients may make mistakes.

We believe that physical therapist (PT) students, in the course of their professional training, are often provided with loosely defined rules for categorizing patients into diagnostic categories. In clinical practice, they gain experience in applying these rules to patients. As a result, experienced physical therapists may have not only rules for diagnoses but also memories of patients for whom different rules were applicable. Our research examines whether remembering patients affects the ability of physical therapists to correctly apply rules for diagnostic categorization.

Our research is based on a series of categorization experiments reported by Allen and Brooks.[16] They examined the conditions under which application of a simple rule for categorization is affected by prior experience with specific stimuli. In the first experiment of their series, Allen and Brooks[16] asked subjects to categorize fictitious animals as either "builders" or "diggers." Half of the subjects were given a simple rule for categorization, and half of the subjects were not given a rule. In the first phase of the experiment, subjects had to categorize each of eight different animals five times. Immediately after making each categorization, the subjects were given the correct category names for the animals. In the second phase of the experiment, subjects had to categorize each of the eight original animals numerous times, but they also had to categorize eight new animals. Based on the investigator-defined five-dimensional category structure, each of the new animals was closest to (in the five-dimensional category scheme) and, therefore, most similar to (differed on only one dimension) only one of the original animals; thus, there were eight original-new, similarity-based pairs of animals. In four of the original-new pairs, the new animal was in the same category as the most similar original animal. In the other four original-new pairs, the new animal was in the opposite category as the most similar original animal.

Allen and Brooks[16] hypothesized that subjects would respond based on similarity to previously seen animals despite the fact that the subjects were given a rule for categorization. The crucial test of their hypothesis was based on differences in the percentage of categorization errors for same-category and opposite-category new animals, as well as original animals. The subjects made more errors on opposite-category new animals than on either same-category new animals or on original animals. Allen and Brooks[16] concluded that subjects made errors on opposite-category new animals because they were responding based on similarities of the new animals to the original animals rather than strictly adhering to the rule for categorization.

Smith and Zarate[17] were also able to elicit errors related to a similarity-based effect in which the stimuli were words and the context in which the words were presented was a paragraph of text. Given the large amount of written information contained in medical records that must be reviewed by clinicians, we believe the possibility that similarity-based diagnostic errors might occur with verbal information warrants investigation. Thus, we wanted to determine whether subjects would make errors consistent with a similarity-based response effect on written descriptions of patients with low back pain presented in the context of a simulated medical record.

We used a categorization task similar to the task used by Allen and Brooks.[16] In our study, however, the fictitious animals were replaced with descriptions of patients with low back pain, and students were asked to diagnose the patient's condition. The primary purpose of our experiment was to test the hypothesis that students would make a higher percentage of errors on opposite-diagnosis new patients than on either same-diagnosis new patients or original patients.

Assuming that memory for previously seen stimuli is related to the occurrence of similarity-based response errors, we included several factors designed to enhance memorability of the individual simulated patient stimuli for some of the students, including different sets of instructions, different types of students, and different types of stimuli. First, students in only two of the three instructional conditions were told to remember as much as possible about the patients. Second, half of the students we studied were first-year PT students, and the other half were non-physical therapist (NPT) students. Because the PT students had not previously studied the low back categorization scheme, they were not familiar with the specific diagnoses used in the experiment. They were, however, familiar with the terminology used and presumably were interested in the content area. The NPT students were not expected to be familiar with or interested in the content area. Thus, we expected that the stimuli would be easier for the PT students than for the NPT students and increase the likelihood of making similarity-based errors for the PT students as compared with the NPT students. Finally, because vividness has been shown to be effective for enhancing memorability,'8 we used the criteria cited by Taylor and Thompson[18] to create vivid descriptions for half of the patient stimuli and pallid descriptions for the other half of the patient stimuli. More similarity-based response errors were expected for cases with vivid patient descriptions than for cases with pallid patient descriptions.

Method

Subjects

The subjects were 60 PT students and 60 NPT students. Prior to participation in the experiment, all students read and signed an informed consent document approved by the Washington University Institutional Review Board. In exchange for their participation in the experiment, the students received either extra credit for the class from which they were recruited or $7.00. The PT students were familiar with the terminology used in the patient cases but not with the specific categorization scheme. The characteristics of the sample in terms of age and gender are summarized in Table 1.

Table 1.
Sample Description

                                          Age (y)

Academic Major(a)            n          X          SD

PT group                     60      24.65[b]      4.64

Men                          18      26.50         5.69
Women                        42      23.86         3.93

NPT group                    60      19.95[b]      1.31

Men                          25      20.20         1.68
Women                        35      19.77         0.94

Total                       120      22.30         4.14

(a) Because many instances of gender effects are documented in the literature, we considered the effects of gender in our study. None of the effects, however, were significant.

(b) PT group=physical therapist students, NPT group=non-physical therapist students.

(c) Significantly different (F=56.98; df=1,118; P [is less than] .001).

Experimental Design

We studied two between-groups factors: academic major and instructional condition.(*) All 60 members of the PT group were PT students at Washington University, St Louis, Mo, and all 60 members of the NPT group were enrolled in an upper-level undergraduate psychology course at Washington University. Twenty students from each level of academic major were assigned randomly to one of three instructional conditions: rule plus (RP), rule only (RO), and no rule (NR). The instructions conditions differed in terms of specifying the information that the students were to use in making the diagnoses. Students in the RO instructional condition were given a simple rule for diagnostic categorizations. Students in the NR instructional condition were not given a rule but were told to remember as much as possible about each patient and to use the information they remembered to help make a diagnosis for subsequent patients. Students in the RP instructional condition were given a rule for diagnostic categorizations and told to remember as much as possible about each patient.

The design also included three within-subject factors: "ON" type, subphase, and "VP" type. The ON type was indicative of whether the written patient case description was originally seen in the first phase of the experiment (old patient case description) or in the second phase of the experiment (new patient case description). The second within-subject factor specified a subphase of the second part of the experiment. In the positive subphase, each new patient case description was in the same diagnostic category as the most similar old patient case description. In the negative subphase, each new case was in the diagnostic category opposite that of the most similar old case. The VP type was related to the type of description used in the patient cases: vivid or pallid.

Instrumentation and Software

The experiment was conducted using a COMPAQ 386/25 DeskPro personal computer and a 35.6-cm (14-in) COMPAQ color monitor with a VGA color adaptor interface card.[dagger] The computer program used to run the experiment was written in QuickBASIC, version 4.5.[double dagger] The program ran without interruption through the instructions, the training phase, and the test phase of the experiment. The primary computer programs used to perform bootstrap analyses were written in MATLAB.[sections]

Stimuli for Simulated Diagnosis Task

The stimuli for the simulated diagnosis task were brief descriptions of fictitious patients who had low back pain that could be categorized as either a flexion syndrome or an extension syndrome. The syndromes are defined in Table 2. For example, patients with a straight back in standing, a slumped sitting posture, and pain when bending over would be categorized as having a flexion syndrome. A total of 16 patient cases were used in the experiment. Each patient case was constructed from two types of elements: (1) a set of five signs and symptoms, as indicated in Table 2, and (2) a brief patient description.

Table 2.
Dimensions and Codes for Signs and Symptoms Used for Patient
Case Stimuli

                            Signs and Symptoms for
                            Diagnostic Syndromes

Relevant                   Extension            Flexion
Dimensions                 Syndrome             Syndrome

Shape of low back in       Curved inward (0)    Straight (1)
standing (SLS)

Painful movement of        Straightening up (0) Bending down (1)
back (PMB)

Painful sitting position   Upright (0)           Slumped (1)
(PSP)

Irrelevant                 Nondiagnostic Values
Dimensions

location of symptoms       Proximal (0)          Distal (1)
(LOS)

Symptoms with light        Pain (0)              Tingling (1)
palpation (SLP)

The unique combination of signs and symptoms for each of the 16 patient cases used as stimuli in the experiment is provided in Table 3. For example, the values used for patient case 1, a flexion syndrome case, were "straight" for the shape of the low back in a standing position, "bending down" for painful movement of the back, "slumped" for a painful sitting position, "proximal" for the location of symptoms, and "pain" for symptoms with light palpation. Using the sets of signs and symptoms for the five dichotomous dimensions, short paragraphs containing the sets of signs and symptoms for each patient case were constructed. In most instances, the paragraphs consisted of three sentences. Except for the substitution of the appropriate signs and symptoms, the same paragraph was used for both members of a pair. Note in Figure 1 that the only difference between the two paragraphs is the inclusion of "bending over" in case 1 and "straightening up" in case 10.

Table 3.
Sign and Symptom Combinations for Patient Case Stimuli(a)

Old Cases

Case No.    SLS    PMB    PSP    LOS    SLP

1            1      1      1      0      0
2            0      1      1      0      1
3            1      0      1      1      1
4            1      1      0      1      0
5            1      0      0      0      1
6            0      1      0      1      1
7            0      0      1      1      0
8            0      0      0      0      0

New Cases

Case No.    SLS    PMB    PSP    LOS    SLP

9            1      1      1      1      1
10           1      0      1      0      0
11           1      1      0      0      1
12           0      1      1      1      0
13           0      1      0      0      0
14           0      0      1      0      1
15           0      0      0      1      1
16           1      0      0      1      0

(a) Abbreviations for dimensions and codes for signs and symptoms are noted in Table 2.

Figure 1.

Case 1

The location of symptoms is proximal. Sitting slumped, light

palpation and bending over produce pain. In standing, the shape

of the low back is straight.

Case 10

The location of symptoms is proximal. Sitting slumped, light

palpation and straightening up produce pain. In standing, the shape

of the low back is straight.

Sample of specific paragraphs used in simulated diagnosis task. Case 1 originally appeared in the training phase, and case 10 originally appeared in the test phase. Feature values are in bold text in the figure but were not presented in bold text for the subjects during testing. Except for use of bold text in the figure, the sample is shown verbatim as presented to the students.

In addition to the five dichotomous dimensions, a brief description of each patient was provided under the heading "Patient Description." The description was not relevant to the patient's diagnosis. Each description included a name consisting of a first initial and last name, as well as a brief statement about the patient. There were two types of descriptions: vivid and pallid. The names for the vivid descriptions were defined as being uncommon for the region (maximum of 5 entries in the metropolitan area telephone book) and contained no more than two syllables. In accordance with the criteria cited by Taylor and Thompson,[18] we attempted to make the names humorous, emotionally interesting, or imagery-provoking. The names for the pallid descriptions were defined as commonly occurring in the region (minimum of 800 entries in the metropolitan area telephone book) and contained no more than two syllables. These names were not famous names and, in our view, were not imagery-provoking or emotionally interesting. In the brief statement about each patient, the number of dimensions addressed varied between two and four. The dimensions from which features were chosen included age, nationality, gender, marital status, socioeconomic status, intelligence, behavior, physical appearance, number of children, and occupation. Samples of the names and statements used in the patient descriptions are shown in Figure 2.

Figure 2.

Name         Description

G Mango      stormed into the out-patient waiting room demanding to
             be seen immediately.

S Kraut's    favorite hobby is creating a batch of prize-winning,
             lip-puckering pickles.

C Won        , age 47, still competes successfully as an
             Olympic-level ping pong player.

H Quigle     is a brilliant espionage agent who works for the CIA in
             Iran.

B Jones      is an elementary school teacher in one of the
             metropolitan area's communities.

E Davis      has taken aptitude tests twice this past year and
             received average scores.

K Baker      is married, has two children and goes to work daily at
             eight.

H Roberts    has an average income and lives in an apartment with
             his wife.

Sample of the names and statements used in the written patient case descriptions for the simulated diagnosis task. The first four sets are examples of vivid names and descriptions; the second four sets are examples of pallid names and descriptions. The sample is shown verbatim as presented to the students.

Two rules for making the diagnoses were used. The rule for the flexion syndrome was the following: If a patient has any two of the three relevant sign or symptom features for the flexion syndrome, that is, slumped, straight, and bending over, the patient is classified as having a flexion syndrome. The analogous rule for the extension syndrome incorporated the following three sign or symptom features: upright, curved inward, and straightening up. One rule was assigned randomly to each student. Half of the students in each of the two rule instructional conditions (de, RP and RO) were assigned the rule for the flexion syndrome, and the other half were assigned the rule for the extension syndrome.

Procedures for Simulated Diagnosis Task

Each student was trained and tested individually in a small, quiet room. After the student read and signed an informed consent form, he or she was asked to attend to the instructions on the computer display. The student read the instructions and, when finished, pressed a key to continue with the next display. The general instructions were the same for all students, but the instructions about making the diagnosis differed according to the student's assignment to a particular instructional condition (de, RP, RO, or NR).

In the training phase, students were required to make diagnoses for eight different patients on five different occasions, resulting in a total of 40 diagnostic judgments. Prior to presentation of the first patient for each student, the computer randomly assigned a unique patient description to each of the 16 different sets of signs and symptoms. Following the linking of a patient description with a set of signs and symptoms, the information for each patient was incorporated into a patient medical record format for presentation to the student.

The eight patient cases randomly selected for use in the training phase were presented to students in a pseudo-random order. The randomization of presentation order was controlled to the extent that all eight patients appeared in each set of eight trials but no patient appeared twice in succession. After the students entered a diagnosis, they were given feedback regarding the correctness of their response and the correct diagnosis for that patient. After each set of eight trials, the students were given summary speed and accuracy information for the preceding eight trials and additional instructions that encouraged them to proceed as quickly as possible without making errors. Immediately after the training phase was completed, the test phase began.

In the test phase, students were required to make diagnoses for eight of the patient cases previously seen and for eight new patients. The test phase was divided into two subphases: positive and negative. In the positive subphase, the four new patients had the same diagnosis as the most similar previously seen patient. In the negative subphase, the four new patients had the diagnosis opposite that of the most similar previously seen patient. In both the positive and the negative subphases, old cases were used as fillers between the new cases. Overall, each of the eight previously seen patients was presented four times and each of the eight new patients was presented once, for a total of 40 trials. Instructions to the students for the test phase reiterated that their task was to make a diagnosis for each patient as quickly as possible without adversely affecting accuracy. In addition, the students were informed that new patients would be presented for diagnosis. No feedback was given in the test phase. At the completion of the test phase, the names of the patients were displayed individually on the computer screen, and students were asked to write down as much as they could remember about the patients. The maximum amount of time that students could use to complete both phases of the experiment was 1 hour.

The primary dependent variable was the percentage of diagnostic errors. Percentages of errors were calculated for each item type (de, opposite-category old, same-category old, opposite-category new, and same-category new). Two additional variables (de, response time and percentage of recall points) were measured and are described below. Response time for a diagnostic categorization, defined as the time elapsed between the keystroke that initiated presentation of the patient case on the computer display and the keystroke that signaled the student's diagnostic response, was measured in milliseconds. Only the mean response times for correct diagnostic responses were used in the analyses because the response times for incorrect responses were uninterpretable. The amount of information recalled about patient cases at the end of the experiment (recall points) was calculated as a percentage of the total of information presented for each case.

Data Analysis

Although the experimental design was compatible with the use of analysis-of-variance (ANOVA) techniques, one of the major assumptions underlying the use of ANOVA techniques is that the data are distributed normally. In our experiment, the data for the dependent variables were not distributed normally. Because the violation of normality is especially problematic with repeated-measures ANOVAs, the validity of inferences based on the results of using ANOVA techniques with the data from this experiment would be suspect. One option for analyzing data that are not distributed normally is the use of a resampling procedure known as the bootstrap procedure.[19,20] The primary advantage of the bootstrap procedure is that it provides a means for testing hypotheses using a sampling distribution based on the sample actually studied, regardless of the shape of the distribution. Consequently, inferences based on bootstrap analyses are less likely to be erroneous than those based on conventional parametric analyses, such as ANOVA.

At a general level of description,[parallel] bootstrap procedures involve (1) generation of a sampling distribution for the variable of interest based on multiple samples drawn from the data set obtained in a given experiment and (2) construction of confidence intervals for the distribution of the variable estimate against which the statistical significance of hypothesized effects could be tested. Consistent with the fixed-effects design of the experiment, a residual resampling technique[21] was used to generate 1,000 bootstrap samples. Then a hierarchical multiple regression analysis was performed on each of the 1,000 bootstrap samples. The median of the distribution of 1,000 regression coefficients for each effect was determined and was used as the best estimate of the coefficient for the effect.

After calculating the bootstrap estimates, we used the percentile t method[18] to construct the 95% confidence intervals for the bootstrap estimates that we used for hypothesis testing at an alpha level of .05. Main and interaction effects were considered significant if the null hypothesis value of zero fell either at or outside the 95% confidence limits.

Currently, no practical method for either conducting post hoc analyses on all possible comparisons or estimating power has been developed to supplement the hypothesis testing capability afforded by the bootstrap procedures. Consequently, the reader should assume that statements made about differences between specific pairs of values are intended only to describe the possible relationship, with the exception of those differences related to effects in which only two means are compared. Conclusions about the reliability of the pair-wise comparisons must be deferred, and the inferences drawn must be considered tentative. Due to constraints imposed by the use of contrast coding for the between-groups effects, it was not possible to test for differences among all three instructional conditions simultaneously. For cases in which students who were told the rule were compared with those who had not been told the rule, the label "RU instructional condition" was used to represent students in both the RP and RO instructional conditions.

Results

Percentage of Diagnostic Errors

Students in the RU instructional condition made a smaller percentage of diagnostic errors (median = 25%) than did students in the NR instructional condition (median = 34%). The two-way interaction between RU and NR instructional conditions and subphase was significant but will not be described in detail because the three-way interaction among the RU and NR instructional conditions, ON type, and subphase was also significant and will be discussed. As indicated in Figure 3, students in the RU instructional condition made fewer or about the same number of errors on new cases as on old cases in both subphases. By contrast, students in the NR instructional condition made more errors on new cases than on old cases in the positive subphase but made fewer errors on new cases than on old cases in the negative subphase. In addition, the difference in percentage of errors for old cases compared with new cases was larger for students in the NR instructional condition than for students in the RU instructional condition.

[Figure 3 ILLUSTRATION OMITTED]

Both the two-way (academic major [PT and NPT] x subphase) and three-way (academic major x ON type x subphase) interaction effects were significant. The graphic representation of the three-way interaction effect (Fig. 4) indicates that the pattern of errors for old cases compared with new cases differed both across groups and across subphases. Students in the PT group made more errors on new cases than on old cases in the positive subphase and fewer errors on new cases than on old cases in the negative subphase; the reverse was true for students in the NPT group. Overall, the smallest percentage of errors was made by PT students on new cases in the negative subphase.

[Figure 4 ILLUSTRATION OMITTED]

Response Time

Students in the RP instructional condition took longer (median = 13,047 milliseconds) to make diagnoses than did students in the RO instructional condition (median = 11,059 milliseconds), and they took longer to make diagnoses on new cases (median = 13,136 milliseconds) than on old cases (median = 11,308 milliseconds). As indicated by the significant two-way interaction depicted in Figure 5, however, the magnitude of the difference in time spent on new cases compared with old cases was much larger for students in the RP instructional condition than for students in the RO instructional condition and the difference in response time between instructional conditions was relatively small for old cases but relatively large for new cases.

[Figure 5 ILLUSTRATION OMITTED]

The three-way interaction among academic major, ON type, and subphase was also significant. Figure 6 shows that students in both groups took longer to make diagnoses for new cases than for old cases in both subphases but that students in the PT group tended to take longer to make diagnoses in all conditions than did students in the NPT group. The difference between PT and NPT students was largest in the negative subphase. Figure 6 reveals that students in the NPT group made slightly faster diagnoses on new cases in the negative subphase than in the positive subphase but that the pattern was reversed for students in the PT group.

[Figure 6 ILLUSTRATION OMITTED]

Recall Points

Students in the RP instructional condition recalled more about cases (median = 20%) than did students in the RO instructional condition (median = 13%). The students recalled more about cases with vivid patient descriptions (median = 21%) than about cases with pallid patient descriptions (median = 11%). They recalled more about the patient description (median = 25%) than about all other parts of the case (median = 13%), less about signs and symptoms (median = 6%) than about diagnoses (median = 26%), and more about relevant signs and symptoms (median = 8%) than about irrelevant signs and symptoms (median = 5%).

Discussion and Conclusion

Contrary to expectations, there was no difference in the percentage of diagnostic errors made between the PT and NPT groups, but there were differences between the groups in terms of percentage of errors on old patient case descriptions compared with new patient case descriptions across the two subphases (Fig. 4). The behavior of students in the PT group for the positive subphase was consistent with a practice effect for the old patient case descriptions. We expected more errors for new patient case descriptions because they would be new and would not have been practiced with corrective feedback. The increase in percentage of diagnostic errors for new patient case descriptions compared with old patient case descriptions in the negative subphase for the NPT group was consistent with a similarity-based response effect, but the increase was very small compared with the analogous difference noted by Allen and Brooks[16] for drawings of fictitious animals. This increase, however, was similar in magnitude to the difference reported by Allen and Brooks for lists of words. In addition, contrary to expectations, students in the PT group made fewer errors on new than on old patient case descriptions in the negative subphase. Smaller percentages of errors on new patient case descriptions compared with old patient case descriptions in the negative subphase would be consistent with the subjects recognizing the new patient case descriptions as being not only new but also different from the new patient case descriptions in the positive subphase and taking extra time to ensure making the correct diagnosis. The response time data tend to support the conclusion that, especially in the negative subphase, the PT students emphasized accuracy instead of speed, whereas the NPT students emphasized speed rather than accuracy.

As we expected, students who were given a rule to follow made a smaller percentage of diagnostic errors than did students who were not told the rule, and the two groups behaved differently in the subphases. In terms of the similarity-based response effect, however, the most important data came from subjects in the RU instructional condition. The RU instructional condition included students in both the RP and RO instructional conditions. Contrary to our expectations, students in the RU instructional condition made fewer or about the same number of errors on new patient case descriptions compared with old patient case descriptions in both positive and negative subphases, as indicated in Figure 3. The results are not consistent with a similarity-based response effect, but they are consistent with students recognizing that the new patient case description is new and taking extra care to make the correct diagnosis.

Our study provides little support for the occurrence of similarity-based response errors during the simulated diagnostic task in which the stimuli were written descriptions of the patients. In a recent study by Neal et al,[22] however, tape-recorded descriptions of hypothetical loan applicants coupled with visual inspection of photographs of the applicants yielded similarity-based response errors. Clinicians' encounters with patients require both verbal exchange and visual observation. We therefore do not believe that it is safe to assume that clinicians do not make similarity-based response errors.

As we expected, students in the RP instructional condition took longer to make diagnoses than did students in the RO instructional condition. Presumably, students in the RP instructional condition were attempting to comply with the instruction to remember as much as possible about the patients in addition to making the diagnosis. Based on overall estimates for the two instructional conditions, the mean response time for subjects in the RO instructional condition ([bar] X = 11,059 milliseconds) differed by about 2 seconds from the mean response time for students in the RP instructional condition ([bar] X = 13,047 milliseconds). The two-way interaction effect shown in Figure 5 indicates that the groups differed most in the amount of time spent on new patient case descriptions (ie, a difference of 2.78 seconds). The size of the difference appears to reflect a substantive difference in the behavior of students under the two instructional conditions.

The academic major of the students had an effect on the differences in the amount of time they took to respond on old patient case descriptions compared with new patient case descriptions in the positive and negative subphases. Students in the PT group took longest to make diagnoses on new patient case descriptions in the negative subphase. The increase in time required to make diagnoses on new patient case descriptions compared with old patient case descriptions in the negative subphase was larger for students in the PT group than for students in the NPT group, and students in the NPT group took slightly less time to diagnose new patient case descriptions in the negative subphase than in the positive subphase. Because the interaction effect does not include the instructional condition, it is difficult to interpret the findings in relation to the existence of a similarity-based response effect, but it does appear that students in the PT group sacrificed speed for accuracy to a greater extent than did students in the NPT group. The longer response times for each condition for students in the PT group suggests that familiarity with the terminology used in the case descriptions had less of an effect on the performance of students in the PT group than did their presumed interest in diagnostic accuracy, but these elements were not measured.

Students recalled more about cases with vivid patient descriptions than those with pallid patient descriptions. Approximately 20% of the information about cases with vivid patient descriptions and about 10% of the information about cases with pallid descriptions were recalled. Although approximately twice as much information was recalled about cases with vivid patient descriptions compared with those with pallid patient descriptions, the overall percentage of information recalled was small. In addition, the students had less difficulty remembering the diagnosis for a patient than the collection of signs and symptoms for a patient. The fact that students recalled very little information about signs and symptoms suggests that the similarity between investigator-designated pairs of old and new patient case descriptions based on the assigned sets of signs and symptoms probably was not apparent to the students.

Implications for Developing Strategies to Minimize Diagnostic Errors

An overall purpose of our study was to increase our understanding of the psychological processes that underlie the formation of clinical diagnostic judgments so that strategies aimed at minimizing errors can be developed. Although the importance of providing a rule for diagnosis was not a primary focus of the experiment, it is clear from the results of this study that students who had been given a rule for categorization made fewer categorization errors than did students who had not been given a rule. As obvious as this finding might appear, we believe that clinicians seldom have adequate rules to use. Thus, one way clinicians may be able to minimize their diagnostic errors is to search for, assist in the development of, and systematically use valid rules for diagnosis.

Based on the results of this study, it does not appear that reading information about a patient leads to the production of erroneous similarity-based diagnostic responses by students who are preparing to become PTs and who have been given a rule for diagnosis. Quite to the contrary, the PT students were most accurate on the critical opposite-category new patient case descriptions. Considering that they took longer to make diagnoses on the critical test patient case descriptions than on related old patient case descriptions, it appears they were aware that the critical test patient case descriptions represented patients with different sets of signs and symptoms than the similar old patients and took extra time to make the correct diagnoses. Thus, we believe the diagnostic accuracy of PT students could be enhanced by provision of rules for diagnosis and adequate time to apply the rules throughout their curriculum, but our data do not directly address this issue. It is also clear that vivid information was recalled better than pallid information. Consequently, we believe it may be useful for faculty to incorporate vivid information in their description of critically important cases. Data supporting this contention, however, are not yet available, and further research to test this hypothesis is needed.

Although our subjects were students, we believe that the results of our study have implications for experienced clinicians. Visual, nonlinguistic stimuli were not included in the current experiment, so the findings do not contradict the results of previous studies in which visual stimuli were used. In clinical settings, however, a host of visual, auditory, palpatory, and linguistic stimuli can be associated with each patient. Thus, we believe it is inappropriate to conclude that clinicians will completely avoid diagnostic errors that may be the consequence of reliance on the similarity of new patients to previously seen patients. None of the subjects in our study were experienced clinicians. Given the time pressure of actual practice and the practicing clinician's presumed store of clinical experiences, the incidence of similarity-based responding could be higher for experienced clinicians than for students. Similarity-based responding, as well as chance responding, could result in a correct diagnosis, but either process could also result in an error. Until evidence to the contrary is presented, it is safe to assume that systematic comparison of each patient's signs and symptoms to rules for diagnosis will minimize diagnostic errors to a greater extent than either reliance on a patient's similarity to previously seen patients or chance.

Acknowledgments

We acknowledge the many thought-provoking contributions of David Balota, PhD, and Alan Lambert, PhD, to the development of the experiment and the interpretation of the results.

(*) A third factor yielded no important significant results and will not be reported in this article.

[dagger] COMPAQ Computer Corp, 20555 FM 149, Houston. TX 77070.

[double dagger] Microsoft Corp, One Microsoft Way, Redmond, WA 98052-6399.

[sections] The MathWorks Inc. Cochituate Place, 24 Prime Park Way, Natick, MA 01760.

[parallel] For a full description of bootstrap procedures, see Dalgleish[19] and Mooney and Duval.[20]

References

[1] Rose SJ. Description and classification: the cornerstones of pathokinesiological research. Phys Ther. 1986;66:379-381.

[2] Rose SJ. Editor's note: Musing on diagnosis. Phys Ther. 1988;68:1665.

[3] Rose SJ. Physical therapy diagnosis: role and function. Phys Ther. 1989;69:535-547.

[4] Sahrmann SA. Diagnosis by the physical therapist: a prerequisite for treatment. Phys Ther. 1988;68:1703-1706.

[5] Guide to Physical Therapist Practice. Phys Ther. 1997;77:1163-1650.

[6] Arkes HR, Hammond KR, eds. Judgment and Decision Making: An Interdisciplinary Reader. Cambridge, England: Cambridge University Press; 1986.

[7] Goldberg LR. Simple models or simple processes. Some research on clinical Judgments. In: Arkes JR, Hammond KR. eds. Judgment and Decision Making: An Interdisciplinary Reader. Cambridge, England: Cambridge University Press: 1986:335-353.

[8] Meehl PE. Clinical Versus Statistical Prediction: Theoretical Analysis and a Review of the Evidence. Minneapolis. Minn: University of Minnesota Press: 1954.

[9] Dowie J. Elstein AS, eds. Professional Judgment: A Reader in Clinical Derision Making. Cambridge, England: Cambridge University Press; 1988.

[10] Elstein AS. Clinical reasoning in medicine. In: Higgs J, Jones MA, eds. Clinical Reasoning in the Health Professions. Oxford, England: Butterworth-Heinemann Ltd; 1995:49-59.

[11] Hayes BK, Adams RD. Parallels between the process of clinical reasoning and categorization. In: Higgs J, Jones MA, eds. Clinical Reasoning in the Health Professions. Oxford, England: Butterworth-Heinemann Ltd; 1995:147-156.

[12] Jones MA. Clinical reasoning in manual therapy. Phys Ther. 1992; 72:875-884.

[13] Norton BJ. Clinical decision making in physical therapy practice. In: Myers RS, ed. Saunders Manual of Physical Therapy Practice. Philadelphia, Pa: WB Saunders Co; 1995;17-36.

[14] Patel VL, Arocha JF. Methods in the study of clinical reasoning. In: Higgs J, Jones MA, eds. Clinical Reasoning in the Health Professions. Oxford, England: Butterworth-Heinemann Ltd; 1995:147-156.

[15] Brooks LR, Norman GR, Allen SW. Role of specific similarity in a medical diagnostic task. J Exp Psychol Gen. 1991;120:278-287.

[16] Allen SW, Brooks LR. Specializing the operation of an explicit rule. J Exp Psychol Gen. 1991;120:3-19.

[17] Smith ER, Zarate MA, Exemplar and prototype use in social categorization. Social Cognition. 1990;8:243-262.

[18] Taylor SE, Thompson SC. Stalking the elusive "vividness" effect. Psychol Rev. 1982 ;89: 155-181.

[19] Dalgleish LI. Discriminant analysis: statistical inference using the jackknife and bootstrap procedures. Psychol Bull. 1994;116:498-508.

[20] Mooney CZ, Duval RD. Bootstrapping: A Nonparametric Approach to Statistical Inference. Newbury Park, Calif: Sage Publications Inc; 1993.

[21] Hamilton LC. Regression With Graphics: A Second Course in Applied Statistics. Pacific Grove, Calif: Brooks/Cole Publishing Co; 1992.

[22] Neal A, Hesketh B, Andrews S. Instance-based categorization: automatic versus intentional forms of retrieval. Memory & Cognition. 1995;23:227-242.

BJ Norton, PhD, PT, is Associate Director for Postprofessional Studies, Program in Physical Therapy, Washington University School of Medicine, 4444 Forest Park Blvd, St Louis, MO 63108 (USA) ([e-mail]). She was a candidate for the Doctor of Philosophy degree in psychology at Washington University when this research was conducted in partial fulfillment of her degree requirements. Address all correspondence to Dr Norton.

MJ Strube, PhD, is Professor of Psychology, Washington University.

This study was approved by the Washington University Institutional Review Board.

This research was presented, in part, at the Combined Sections Meeting of the American Physical Therapy Association, February 12-16, 1997, Dallas, Tex, and at Physical Therapy '97: the American Physical Therapy Association's Scientific Meeting and Exposition, May 30-June 2, 1997, San Diego, Calif.

This article was submitted January 15, 1997, and was accepted June 27, 1997