Case conceptualization and treatment planning: investigation of problem-solving and clinical judgment.

This investigation examined the cognitive factors that influence case conceptualization (CC) and treatment planning (TP) tasks among experienced mental health professionals. A thinking aloud process-tracing strategy was used to identify problem-solving styles and clinical judgment strategies used by 25 licensed psychologists, clinical mental health counselors, and clinical social workers while responding to a standardized case conceptualization and treatment planning task. Cluster analysis revealed a four cluster solution that differentiated among treatment planning scores of these clinicians. SPSS discriminant analyses identified (a) three problem-solving styles (i.e., differentiation, integration, affiliation) that correctly predicted cluster membership in 96% of cases, and (b) three clinical judgment strategies (i.e., minimal, complex, heuristic) that correctly predicted cluster membership for all of these clinicians. Implications of these findings for training and research are presented.

**********

Case conceptualization and treatment planning are frequent and universal clinical judgment tasks of mental health practitioners (Benbenishty & Treistman, 1998; Falvey, 2001; Garb, 1998; Prieto & Scheel, 2002; Strohmer & Leierer, 2000; Yennie, 1997). How clinicians elicit client information, weigh the value of that input, formulate hypotheses, and utilize cognitive schemas to inform their understanding of clients and their treatment needs has been the subject of an extensive empirical literature in cognitive psychology over the past 50 years. Unfortunately, that research has failed to lessen the considerable variability that exists between what is known about clinical decision-making and what is practiced. In fact, it has been suggested that experienced clinicians may be subject to more rather than less bias in their judgments than are novices (Strohmer & Leierer). Biased judgments are unacceptable among all health-related disciplines, as third party accountability standards increasingly demand evidence-based rather than intuitive or apprenticeship-based decision approaches (Chessare & Lieu, 1998). Educators, clinicians, and supervisors would benefit from a science of clinical reasoning to identify and improve decision-making processes during assessment and treatment planning tasks.

The literature on clinical judgment and information processing has provided some consensus regarding decision-making under uncertainty. For example, it is evident that human information-processing capacity is quite limited. The mind uses a variety of heuristics (i.e., cognitive shortcuts) to handle the information overload that is characteristic of complex judgments. These heuristics tend to reduce the complexity of problems by assessing probabilities based on a limited number of variables across many cases at the expense of considering innumerable variables germane to one individual.

Large caseloads and limited time for assessment and treatment planning would seem to favor such cognitive shortcuts. However, their impact on clinician performance remains unclear.

Studies support the prevalence of a handful of heuristics as potentially powerful explanatory tools in clinical judgment research (Garb, 1998; Hogarth, 1987; Moore, Smith, & Gonzalez, 1997). One common heuristic includes representativeness: assessing the probability that an event or symptom (e.g., reported hopelessness) belongs to a category (e.g., major depressive disorder) by the degree to which it reflects prototypical features of that category (Kahneman & Tversky, 1982). Although this heuristic permits a rapid matching of symptoms to diagnoses, it neglects population base rates and can lead to fundamental attribution errors (Nisbett & Ross, 1980). Another common heuristic is confirmatory bias, in which clinicians seek information that supports their initial hypothesis while ignoring data that may refute that hypothesis (Arkes, 1991). This heuristic simplifies the complexity of information gathering but may lead to premature foreclosure of diagnostic and treatment decisions. Other heuristics commonly found to contribute to inferential errors in clinical judgment include availability (i.e., the ease with which similar cases or events are recalled), illusory correlation (i.e., correlating traits and symptoms based on personal belief in the absence of objective criteria), primacy effects (i.e., rapid judgments based on very little data), and anchoring effects (i.e., influence attributable to the order of presentation of information on judgments) (Butcher & Schofield, 1984; Dawes, 1994; Kleinmuntz, 1990; Strohmer & Leierer, 2000; Tutin, 1993). While professional counselors acknowledge that heuristics provide necessary and often useful templates for clinical decision-making, a surprising number of clinicians, even experts, are unable to verbalize the presence or potential impact of these specific strategies on their decisions (Garb).

Research has also established that clinicians use organized knowledge structures (i.e., schemas) to process information (Anderson, 1996; Mahoney, 1998; Prieto & Scheel, 2002). Schemas may be based on some theoretical standard (i.e., clinical orientation), formal decision aids (i.e., DSM criteria), empirical evidence, or clinical experience. The validity of these schemas is obviously critical to the accuracy of judgments. Unfortunately, to a surprising degree, clinicians?even experts?are unable to report what schemas they use in their judgment processes (Faust, 1986; Garb, 1998). Research has failed to make explicit the intervening stages in clinical problem-solving styles that mediate between client data and outcome predictions (Strohmer & Leierer, 2000).

These findings leave educators responsible for clinical training and practice in a quandary. To the extent that practitioners cannot identify the judgment heuristics and cognitive schemas on which they base decisions and therefore, lack consistent feedback on the outcomes of those decisions, it seems unlikely that even experts will (a) learn from their experience, (b) transfer effective schemas to future problem-solving tasks, or (c) impart these schemas to mental health trainees. In fact, a conclusion of this extensive area of research is that experience alone does not improve clinical judgment (Dawes, 1994; Garb, 1998; Turk & Salovey, 1988).

In response to the failure of experience to predict judgment outcomes among mental health professionals, studies have attempted to make explicit the cognitive factors that may contribute to performance on judgment tasks. These studies have examined factors such as patterns of inquiry, how clinicians select and synthesize client information (Falvey, 2001); debiasing strategies in diagnostic decision-making (Hill & Ridley, 2001); and cognitive complexity, the ability to integrate large amounts of multidimensional information (Spengler & Strohmer, 1994). Another construct that has been examined is integrative complexity (Ladany, Marotta, & Muse-Burke, 2001; Suedfeldt, Tetlock, & Streufert, 1992), which consists of two components: differentiation, the ability to consider alternative perspectives regarding a phenomenon, and integration, the ability to develop complex connections among differentiated components.

Based on the above body of research, cognitive strategies that have demonstrated potential for improving judgment outcomes include (a) needing to simultaneously consider several alternative diagnoses, explanations, and treatment plans; (b) addressing environmental as well as internal factors influencing client behaviors; (c) decreasing reliance on memory alone in decision-making tasks; and (d) using formal decision aids such as diagnostic criteria, norms, and base rates to improve accuracy (Arkes, 1991; Falvey, 2001; Gambrill, 1990; Garb, 1998; Turk & Salovey, 1986). These findings offer general guidelines to assist in training mental health clinicians in case conceptualization and treatment planning.

Building upon this knowledge base, the current investigation seeks to provide more specificity regarding the cognitive factors that influence case conceptualization (CC) and treatment planning (TP) tasks. Using an intensive process-tracing design and a standardized mental health case, we evaluated transcribed text from a thinking aloud procedure and structured follow-up interviews, as well as written conceptualizations and treatment plans for the case. This design permitted clinician performance to be classified into (a) problem-solving styles (i.e., how clinicians gather and evaluate case information) and (b) clinical judgment strategies (i.e., how clinicians make judgments based on case information). Process variables related to the realistic constraints of evaluating clients and planning treatment that mental health professionals face in their daily clinical practice (e.g., time spent on task components, number of alternative diagnoses considered) were also addressed in the design of this study. Four research questions that guided this exploratory study included the following:

1. What impact does cognitive focus during the case review process have on case conceptualization (CC) and treatment planning (TP) tasks?

2. What problem-solving styles are evident during the case review, and how do they effect CC and TP?

3. What clinical judgment strategies best characterize clinician approaches to CC and TP?

4. Can CC and TP scores be predicted by a classification scheme that distinguishes among problem-solving styles and clinical judgment strategies?

METHOD

Participants

Twenty-five clinicians, nominated by professional mental health colleagues for their specialized work with children, agreed to participate in this study. The sample included 13 females and 12 males with a mean age of 45.1 years (SD = 10.5), and an average of 13 years of clinical experience (SD = 4.9). All participants were Caucasian, licensed either as psychologists (n = 10), clinical mental health counselors (n = 7), or clinical social workers (n = 8). Primary clinical orientations were reported as psychodynamic (n = 10), cognitive-behavioral (n = 6), family systems (n = 7), or humanistic (n = 2). Work settings included inpatient (n = 1), community mental health (n = 8), and private practice (n = 16). Of these clinicians, 19 (76%) reported in follow-up interviews that this standardized case was very similar to actual cases they encountered in their practice; the other 6 participants reported it was fairly similar.

Instruments

The Clinical Treatment Planning Simulation: Case 1-B (CTPS; Falvey, 1994) was one of two instruments used in this investigation. This simulation, which is one of four CTPS cases, represents a client with ADHD, a common clinical disorder among youth (APA, 2000). The case was constructed with input from an interdisciplinary peer-nominated panel of 33 mental health experts selected nationwide. Panelists were drawn from six disciplines that included clinical mental health counselors, psychologists, psychiatrists, social workers, school counselors, and family counselors. Panelists demonstrated their expertise with ADHD by documenting extensive clinical or research credentials pertaining to this disorder. The case was constructed using a Delphi technique which included a series of sequential questionnaires where panelists (a) identified and (b) subsequently ranked all information included in the case file, then (c) identified case conceptualization and treatment planning criteria, and finally (d) ranked these criteria regarding their importance and priority in assessment and planning treatment for this case

Criteria in this CTPS case include all DSM-IV-TR (APA, 2000) diagnostic criteria for ADHD within six sealed reports (e.g., intake interview, teacher evaluations) that also provide a representative psychosocial history, academic and guidance records, medical history, parent interview, and client interview summaries. Respondents are instructed to review the case materials and then subsequently write a case conceptualization (CC) and treatment plan (TP) for the case, as if it were assigned to them. In order to identify their problem-solving styles and clinical judgment strategies, participants were also requested to think aloud while reviewing the case file.

Evaluating CTPS responses relies upon a weighted scoring procedure. Raters identify which of 71 possible coding variables (i.e., 20 CC variables and 51 TP variables) identified by the expert panelists are included in a clinician's written response to this case. Computer scoring is then used to score each variable according to priorities assigned by the experts. Every variable has both a positive weight, calculated as the proportion of experts who ranked that variable as essential, important, or useful, and a negative weight, calculated as the proportion of experts who rated that variable as essential. A respondent's score is the sum of positive weights (i.e., for selected variables) minus negative weights (i.e., for omitted variables) among all 71 coding variables. Using this approach, respondents can be evaluated for their good clinical decisions (i.e., including variables supported the experts) as well as their poor clinical decisions (i.e., omitting variables considered essential by the experts).

Based on this procedure, the CTPS yields two scores: Case Conceptualization (CC) and Treatment Planning (TP). The CC score includes the following variables: (a) initial assessment sources: specific interviews, reports, and referrals reviewed or requested to gain familiarity with the case; and (b) relevant history and status: current symptoms, client presentation, and salient aspects of developmental history. The maximum range of CC scores is 2875, from -1081 if no variables included to +1794 if all variables included. The TP score includes the following variables: (a) clinical considerations: expected format, frequency, and duration of treatment; client prognosis; and anticipated dynamics; (b) treatment interventions: clinical, medical, and environmental interventions as well as client referrals; and (c) treatment goals: targeted aspects of change in the clients' intrapsychic, interpersonal, family, and academic functioning. The maximum range of TP scores is 6017, from -2030 if no variables included to +3987 if all variables included.

The CTPS response form also includes a 10-item evaluation scale on which respondents rank the case format (e.g., clarity, adequate to express experience), fidelity (e.g., comprehensiveness, realism), and usefulness (e.g., for pre-service teaching or in-service evaluation) on a five-point Likert scale (1 = not at all; 2 = slightly; 3 = moderately; 4 = highly; 5 = extremely). CTPS cases have demonstrated content and construct validity in several studies (Falvey, 1992; Falvey, 2001). Further information on CTPS case development, coding and scoring is described in Falvey and Hebert (1992).

The Structured Follow-up Interview (SFI) was developed for this study by the first two authors. It contains 11 questions inquiring about how the respondent arrived at a diagnosis (e.g., confidence in diagnosis, information supporting diagnosis, base rates of disorder, alternative disorders considered) as well as 15 questions regarding influences on treatment planning (e.g., most influential case material, rank order of treatment priorities, prognosis, impact on approach of clinician demographics such as training, gender, orientation, work setting, cultural identity, age, or experience). Scaled responses (e.g., demographic influences ranked as minimal, moderate or significant) were used where possible to permit comparison across cases. The SFI was pilot tested among 10 graduate students in a mental health counselor training program and subsequently modified for clarity. The SFI was administered to each participant by the interviewer immediately following completion of the written tasks.

Procedures

A total of 37 nominees, identified through personal contact with mental health professionals at community agencies and in private practice in a northern New England state, were contacted by phone during a three-month period. Clinicians were invited to participate in a study of clinical decision-making processes. The study was described as follows: (a) they would be instructed to think aloud in the presence of an interviewer while reviewing case material regarding an outpatient childhood disorder; (b) they would next complete a written case conceptualization and treatment plan for the case; and (c) the interviewer would then conduct a structured interview in which they would be asked to recall influences on their decision processes during these tasks. It was noted that both interviews would be audiotaped for later transcription and analysis, and clinicians were asked to schedule a two hour time period for the study.

Of the 37 nominees, 25 clinicians (68%) agreed to participate in the study. Interviews were conducted on-site at each participant's work setting with the interviewer present. No participants had any prior contact with the interviewer. During the process-tracing phase, clinicians were asked to verbalize their thinking processes as they read from among six reports in the CTPS case file. Participants would choose a report, break the seal and read its contents. They were instructed to read as many or as few reports as they felt they needed. The interviewer provided one of the following four minimal prompts if a clinician was not spontaneously thinking aloud: "Why did you choose that report?"; "What seems important to you as you read the report?"; "What have you learned about the case from this report? " and "What further information do you want regarding this case?" The interviewer unobtrusively recorded how long participants took to read each report, as well as inquiry time (i.e., time to identify correct diagnosis) and confirmation time (i.e., time spent further exploring the case material after identifying the correct diagnosis).

Upon completing the case review, participants were instructed to write a case conceptualization and treatment plan using any format they were familiar with in their clinical setting. This open-ended instruction was used to minimize cueing effects on the written tasks. The interviewer left the room during the writing portion of the study. When participants indicated they were done, the interviewer returned to administer the SFI.

Total time for participation ranged from one and a half to three hours. One clinician required two meetings to complete the project, and one clinician was unavailable to complete the follow-up interview due to their time constraints. Each clinician was paid a $25 honorarium for his or her participation in this research.

RESULTS

All written CTPS responses were independently coded by two trained raters. Raters were licensed psychologists, each documenting over 18 years of experience as educators, practitioners, and researchers. Both raters were trained in content analysis and had participated in pilot testing the CTPS coding key. They had no identifying information regarding individual respondents during the coding process. Inter-rater reliability across the 25 responses, computed using the point-by-point statistic (Kelly, 1977), was r = .88. Computer scoring was used to derive a CC and TP score for each respondent from their coded responses.

CTPS evaluation forms were completed by 20 respondents (80%) and yielded ratings for ten items: clarity of instructions (M = 4.15, SD = .75), clarity of format (M = 4.35, SD = .67), tasks adequate to represent clinical experience (M = 4.05, SD = .94), realistic simulation (M = 4.20, SD = .89), comprehensive case file (M = 3.35, SD = .59), useful for teaching (M = 4.35, SD = .59), useful for clinician self-assessment (M = 4.10, SD = .79), useful for competency assessment in credentialing process (M = 4.20, SD = .77), and degree of difficulty (M = 3.0, SD = .65).

Qualitative as well as quantitative analyses were used to explore audiotaped thinking aloud transcripts. The QSR NUD.IST Index System (Scolari, 1996) provided a framework for analysis of the transcribed case reviews and follow-up interviews. Text analysis created a tree-structured index of interrelated concepts that included 3 roots (i.e., conceptual themes) containing 9 branches (i.e., linked categories) and 43 nodes (i.e., subcategories). Conceptual themes included (a) focus of case review, (b) problem-solving styles, and (c) clinical judgment strategies. All thinking aloud transcripts and follow-up interviews were independently evaluated by the trained raters using categories identified in this indexed database and a content analytic procedure (Suedfeld et al., 1992). Inter-rater reliabilities were as follows: focus of case review (r = .90), problem-solving styles (r = .83), clinical judgment strategies (r = .87), and overall reliability r = .86.

Focus of Case Review

The first research question addressed cognitive focus; what case file material clinicians paid particular attention to during their review, that is, what was the impact of cognitive focus on case conceptualization and treatment planning? The conceptual theme Focus of Case Review included four branches; client presentation, relevant history, sources of input, and initial evaluation, which contained 17 nodes. Reviewing transcribed text, raters used a five-point Likert scale to rank participants' cognitive focus for each node (0 = not a focus, 1 = minimal attention, 2 = some impact, 3 = strong impact, 4 = primary focus). A summary of means and predominant cognitive focus for each node is presented in Table 1. Stepwise linear regression analysis of the Focus of Case Review variables revealed no significant predictors of CC scores. TP scores were predicted by two variables: client strengths in step 1 ([beta] = .479, adj[R.sup.2] = .17, F = 5.96, p < .05) and symptoms present in the home in step 2 ([beta] = -.425, adj[R.sup.2] = .33, F = 6.90,p < .001), which together accounted for 33% of variance in treatment planning scores.

For the Client Presentation branch, Symptoms Present in the Home represented the greatest influence on clinician focus during the case review process, with 24 respondents (96 %) attending to this input as a primary or strong focus of their case review. Symptoms Present in the School were also closely examined by most respondents (n = 22; 88 %). These two variables yielded the highest mean ratings among all cognitive focus nodes and were found to be highly correlated (r = .53; p < .01). Client Strengths (n = 12; 48%) were a strong focus for about half of the sample; however, only 3 respondents (12%) were focused on input from the client's Intake Presentation. The emphasis among participants was predominantly on the symptom picture as reported by parents and teachers in their ongoing relationships with this client, rather than from the client's perspective.

For the Relevant History branch, Medical History, Precipitant of Current Referral, and Family or Developmental History were focused on as important aspects of this client's history for over half of these clinicians (ns = 14; 56%). The influence of Academic/School History was less prominent, with nine respondents (36%) emphasizing this variable during their review process and five (20%) paying little or no attention to it.

For the Sources of Input branch, Parent Input was a strong or primary focus for most participants (n = 23; 92%), as was Teacher Input (n = 19; 76%). These two variables were also found to be highly correlated (r = .53; p < .01). This finding is consistent with the cognitive emphasis, reported above, on current symptoms in the home and at school. Counselor Input was a central focus in the case review for about half of this sample (n = 12; 48%), while Client Input was mixed, with equal numbers (n = 9; 36 %) viewing this as an important focus or considering it only minimally.

For the Initial Evaluation branch, eleven respondents (44%) focused strongly on Prior Interventions by teachers and guidance staff interacting with this client in their review of this case. The impact of Formal Academic Testing was mixed, with nine (36%) strongly focused on this variable and eleven (44%) not attending to it. Minimal to no focus was directed at Risk Assessment (e.g., evidence of abuse, substance abuse, or deviant behavior) among a quarter of this sample. Consistency Across Perspectives in the case file was strongly emphasized by five (20%) of respondents during their case review.

Problem-Solving Styles

The second research question addressed problem-solving styles used by mental health clinicians on these CTPS tasks. The conceptual category, Cognitive Schemas, included two branches, client assessment and treatment planning, containing 13 nodes that differentiated problem-solving approaches. Raters evaluated respondents' cognitive schemas from transcribed text on a four-point Likert scale (1 = not at all characteristic to 4 = highly characteristic). SPSS discriminant analysis classified the 13 schemas into three problem-solving styles. Summary ratings and discriminant function coefficients for each node are presented in Table 2. Stepwise linear regression analysis of these 13 cognitive schema variables revealed no significant predictors of CC scores. TP was predicted in Step 1 by Synthesizes Input into Internal Framework ([beta] = .403, adj[R.sup.2] = .13, F = 4.47, p < .05) and in Step 2 by Relies on Own Knowledge of Disorder ([beta] = .412, adj[R.sup.2] = .23, F = 4.72,p < .05), which together accounted for 23% of treatment planning score variance.

For the Client Assessment branch, cognitive schemas that were highly characteristic of a majority of respondents during the case review included Seeks Support for (n = 14; 56%) or Seeks to Disconfirm (n = 17; 68%) Initial Hypotheses; Seeks Direction or Assessment from Others (n = 13; 52%); Synthesizes Input into Internal Framework (n = 14; 56%); Specifies Need for Formal Evaluation (n = 16; 64%); and Reviews All Available Information (n = 13; 52%). Ten respondents (40%) were found to frequently Consider Hypotheses and their Rationales during the review process. Seven respondents (28%) Expressed Affiliation for the Client's Dilemma, while a larger number (n = 11; 44%) did not verbalize an empathic connection with this child.

For the Treatment Planning branch, cognitive schemas that were highly characteristic of a majority of the clinicians in planning treatment included emphasizing a Collaborative Treatment Approach (n = 16; 64%); Relies on Own Knowledge of Disorder (n = 15; 60%); and Anticipates Interventions During Case Review (n = 13; 52%). About one quarter of these respondents emphasized either a diagnosis-based or a client-based approach to treatment, while over half of the sample did not focus on a client-based treatment approach.

SPSS discriminant analysis classified the above 13 cognitive schema nodes into 3 canonical discriminant functions (see Table 2). Integration was represented by verbalized confidence in one's knowledge of the disorder during the SFI follow-up interview, verbal or written evidence of an organized internal cognitive framework to develop complex connections between case material and treatment needs, and a client-based treatment approach. Differentiation was represented by cognitive schemas that reviewed all available information and actively sought input and direction from other perspectives regarding client needs, rather than focusing on supporting or disconfirming one's own hypotheses or anticipating treatment interventions. Affiliation was represented by frequent verbal or written indicators of clinician empathy for the unique aspects of the client's dilemma, and a clear emphasis on a collaborative or team treatment approach.

Clinical Judgment Strategies

The third research question examined clinical judgment strategies for their impact on clinician performance on CC and TP. The conceptual category, Clinical Judgment Variables, included three branches, decision heuristics, data gathering strategies, and cognitive reasoning, that contained 13 nodes. Raters ranked the impact of each node on a three-point Likert scale (1 = minimum; 2 = moderate; 3 = strong) based on thinking aloud transcripts and transcribed SFI interviews. SPSS discriminant analysis classified the 13 clinical judgment variables into three clinical judgment strategies. Summary ratings and discriminant function coefficients for each node are presented in Table 3. Stepwise linear regression analysis of these 13 clinical judgment variables revealed that CC scores were predicted in Step 1 by Verbalizes Awareness of Thought Process (b = .702, adj[R.sup.2] = .24, F = 8.52, p < .01), and in Step 2 by Considers Client Strengths ([beta] = -.393, adj[R.sup.2] = .34, F = 7.08, p < .01), which accounted for 34% of variance in CC scores. No clinical judgment variables predicted TP scores.

For the Decision Heuristics branch, cognitive heuristics commonly found in clinical judgment research (i.e., availability, confirmatory bias, primacy effect, representativeness) were also found among this sample. However, they did not exert a high impact on the case reviews of most of these mental health clinicians. Availability and Representativeness were found to be highly correlated (r = .84; p < .001). One heuristic, the Primacy Effect, had minimal influence on over two-thirds of this sample, suggesting that most respondents avoided premature closure in their review of this case.

For the Data Gathering Strategies branch, Uses Standardized Test Norms (n = 14; 56%) and Seeks Objective Assessments (n = 16; 64%) yielded a high impact on a majority of respondents. These variables were also found to be highly correlated (r = .75; p < .001). Little or no impact was found among a majority for Uses DSM Diagnostic Criteria (n = 16; 64%) or Considers Base Rates of Disorders (n = 20; 80%). Data gathering strategies that Consider Client Strengths yielded mixed influence on respondent judgments, with this variable both highly or not at all impacting about a third of the sample, respectively.

For the Cognitive Reasoning branch, Suspends Judgment During Case Review and Demonstrates Cognitive Complexity highly impacted the clinical judgment of nearly half of this sample (ns = 11; 44%) while reviewing case material. Ten respondents (40%) Verbalized Awareness of their Thought Process during the task. However, despite prompts to think aloud, a similar number of respondents (n = 11; 44%) failed to demonstrate Use of a Causal Reasoning Model to explain their understanding of this case.

SPSS discriminant analysis classified the above 13 clinical judgment nodes into three canonical discriminant functions (see Table 3). A Minimal Strategy reflected the absence of judgment strategies as revealed by very low within-groups correlations between all clinical judgment variables and the discriminant functions. A Complex Strategy was revealed by suspended judgment and the lack of confirmatory bias during the case review process, along with indicators of cognitive complexity in the decision processes. A Heuristic Strategy reflected the lack of verbalized awareness of clinical judgment strategies, the failure to seek objective assessments or evaluate client strengths, and the lack of a causal reasoning model, coupled with the use of DSM diagnostic criteria and some reliance upon heuristics in the decision process.

Classification of Problem-Solving and Clinical Judgment

The final research question asked whether performance on CC and TP tasks could be predicted using a classification scheme that distinguished among problem-solving styles and clinical judgment strategies. First, SPSS hierarchical agglomeration cluster analysis using the average (i.e., between groups) linkage method grouped participants into relatively homogeneous clusters according to CC and TP scores. Squared Euclidean distance coefficients were used as the measure of proximity. There were no cases with missing values, thus all 25 cases were included in the analysis. Although CC scores (M = 1307; SD = 231; range = 762 to 1731) did not usefully distinguish any clusters, TP scores (M = 1184; SD = 763; range = 22 to 2448) generated a four-cluster solution that significantly differentiated scores. Both sexes, both outpatient work settings, and all three mental health disciplines were represented in each cluster. Second, multiple discriminant analysis (Table 4) was used to explore a classification scheme, revealing that TP cluster membership was correctly predicted by (a) three problem-solving styles (integration, differentiation, and affiliation) in 96% of the cases (see Table 2), and (b) three clinical judgment strategies (minimal, complex, and heuristic) in 100% of the cases (see Table 3).

Cluster 1. This cluster of nine respondents was representative of the overall sample in age (M = 44.0 yrs) and clinical experience (M = 13.0 yrs). Three members (33%) reported expertise with ADHD, and five (56%) reported that the case was very similar to cases they treat. During follow-up interviews, these clinicians reported somewhat lower confidence in their diagnosis and a more guarded prognosis (-1 SD) than the overall sample. CC scores for this cluster (M = 1262; SD = 181) ranged near or below the sample mean, while TP scores (M = 824; SD = 179) all fell below the sample mean. These clinicians recorded a longer average inquiry time of 15 minutes, and a shorter average confirmation time of 15 minutes, than the overall sample (i.e., 11 and 23 minutes, respectively). Five respondents (56%) reviewed all reports, and most (n = 7; 78%) considered one or two alternative diagnoses. Eight respondents (89%) were unable to accurately report the base rate for this disorder.

This cluster appears to represent a Template Approach to the TP task (see Table 4). Problem-solving styles of these mental health clinicians included little Differentiation of case input (M = -.15), high Affiliation for the client (M = .76), and the least Integration of case information among any of the clusters (M = -1.54). This cluster was also distinguished by a predominantly Minimal Clinical Judgment Strategy in treatment planning (M = 3.11). These clinicians yielded the highest ratings for Uses Standardized Test Norms and Uses DSM Diagnostic Criteria among the four clusters, thus suggesting an emphasis on the matching of client symptoms to diagnostic criteria. Overall, the Template Approach was most clearly represented by these clinicians' diagnostically driven approach to this clinical task.

Cluster 2. This cluster included four respondents who were similar in age to the larger sample (M = 45.0 yrs) but had significantly less clinical experience (M = 8.5 yrs). None reported expertise with ADHD; as a group they were less familiar (-1 SD) with this disorder than the sample, although three of the four noted the case was very similar to cases they treated. Only one member of this cluster reviewed all six reports; the others reviewed four reports each. In follow-up interviews, they reported less confidence in their diagnosis than the sample as a whole. This cluster's CC scores (M = 1301; SD = 296) were similar to the sample mean but TP scores were extremely low (M = 43.5; SD = 27), with all being more than 2 SD below the sample mean. Recorded process variables revealed a very long inquiry time of 20 minutes (i.e., 1 SD above the sample mean) and a very short confirmation time of 11 minutes compared to the overall sample. Two or three alternative diagnoses were considered, and none of these mental health professionals reported an accurate base rate for the disorder.

This cluster seemed characterized by a Novice Approach to the TP task. Problem-solving among these clinicians noticeably lacked evidence of cognitive schemas involving Integration (M = -1.17), Differentiation (M = -1.18), or Affiliation (M = -1.43). They appeared unable to cognitively evaluate the case information and did not attend to the client's input. In sharp contrast to the Template Approach, they appeared relatively unfamiliar with diagnostic criteria for this disorder. CC and TP scores of these clinicians were similar to trainees, who often score adequately on case conceptualization by including a lot of descriptive information, even though they are unable to synthesize that case material into a working knowledge of how to approach treatment. Clinicians in the Novice Approach employed more heuristic strategies than clinicians in any other cluster, and they were the most effected by the Representativeness heuristic. They were least likely to Use Standardized Test Norms or to Seek Objective Assessments, and they ranked at least one half a SD below the sample means for Demonstrates Cognitive Complexity, Verbalizes Awareness of Thought Process, and Uses Causal Reasoning Model. Overall, the Novice Approach was most clearly represented by the absence of cognitive schemas for problem-solving, with clinicians relying instead on heuristic judgment strategies that resulted in inadequate performance on this clinical task.

Cluster 3. This cluster of five respondents was representative of the overall sample in age (M = 48.5 yrs) and clinical experience (M = 15.0). One member reported expertise with ADHD, though all reported that this case was very similar to cases they treat. This cluster averaged 1 SD above the sample mean for reported confidence in diagnosis and in predicting a positive prognosis for the client. CC scores (M = 1287; SD = 329) ranged widely, from more than 2 SD below to over 1 SD above the sample mean. This cluster was also characterized by extremely high TP scores (M = 2287; SD = 136), all more than one and a half SD above the sample mean. Process variables revealed a short inquiry time of 6 minutes followed by a very long confirmation time of 38 minutes. Average time to complete the written treatment plan (i.e., 58 minutes) for this cluster was 1 SD above the sample mean. Three respondents (60%) reviewed all reports, and an average of three alternative diagnoses were considered. Four respondents (80%) knew the accurate base rate for this disorder.

This cluster seems to characterize a Mastery Approach to treatment planning. These clinicians revealed the most complex clinical judgment strategy (M = 2.33) of the overall sample, and demonstrated the greatest Integration (M = 3.02) of case information in their problem-solving styles. This cluster did not rely on cognitive heuristics; rather, their case reviews were uniformly lengthy and thorough, seeking both to Support and to Disconfirm Initial Hypotheses about the client. Transcribed text revealed that these clinicians were the most likely among the sample to Rely on Own Knowledge of Disorder, Use an Internal Framework to Synthesize Case Input, and Verbalize Awareness of Thought Process. Treatment plans demonstrated a client-based treatment approach and reflected an Affiliation for Client's Dilemma. Overall, the Mastery Approach was most clearly represented by a comprehensive and skilled approach to this clinical task.

Cluster 4. The seven clinicians in this cluster were representative of the overall sample in age (M = 44.0 yrs) and clinical experience (M = 14.0 yrs). Three members (43%) reported expertise with ADHD, though all reported that this case was very similar to cases they treat. During follow-up interviews, these clinicians averaged 1 SD above the sample mean for confidence in their diagnosis and in predicting a better prognosis for this client. This cluster varied widely on CC scores (M=1382; SD = 209), from 1 SD below to one and a half SD above the sample mean. TP scores (M =1513; SD = 270) ranged from average to 1 SD above the sample mean. Process variables revealed a very short inquiry time of 5 minutes and a short treatment planning time of 30 minutes. Five respondents (71%) reviewed all six reports, and an average of three alternative diagnoses were considered. Four respondents (57%) knew the accurate base rate for this disorder.

This cluster seems characterized by an Efficient Approach to these judgment tasks. The clinicians completed thorough yet brief case reviews. Differentiation represented their predominant problem-solving style. More than any other cluster these clinicians tended to Seek Direction from Others to Guide Assessment. Conversely, they were least characterized by the cognitive schemas of Seek to Support or Seek to Disprove Initial Hypotheses and Anticipates Interventions During Case Review. Their clinical judgment was not well represented by any of the identified strategies, though they were less effected by common cognitive heuristics (i.e., Representativeness and Availability) than clinicians in the other clusters. They targeted the diagnosis rapidly, sought direction from others involved in the case, took less time in writing treatment plans, and anticipated a shorter course of treatment than the overall sample. Overall, the Efficient Approach was most clearly represented by clinicians' lack of reliance on clinical judgment strategies and their rapid assessment approach to this clinical task.

DISCUSSION

Results from the process-tracing methodology used in this study provide tentative answers to the four research questions posed. Most intriguing was evidence of distinct, measurable problem-solving styles and clinical judgment strategies among these clinicians. Findings also provide support for a discriminating relationship between those cognitive variables and performance on the treatment planning task.

With regard to the four cluster solution across treatment planning scores, discriminant analyses identified several diverse approaches to these tasks indicating that clinicians analyzed case information in different ways. For example, clinicians using the Template Approach (i.e., low cognitive complexity, lack of differentiation or integration of case information) relied on a straightforward diagnostic matching process to guide their thinking. They were largely unaware of base rates for this disorder, and considered the least number of alternative diagnoses among the sample. By contrast, in the Mastery Approach clinicians used complex judgment strategies and demonstrated the ability to integrate case input into an internal cognitive framework. They provided the most accurate reports of correct base rates for ADHD. Verbal, confident, and thorough, these clinicians averaged the longest time to complete both the CC and TP tasks. Given that the resulting TP scores across these two clusters averaged well over 1 SD apart, this finding supports other evidence for the importance of cognitive complexity in clinical decision-making (Ladany et al., 1992; Spengler & Strohmer, 1994). It also reaffirms the DSM's (APA, 2000) cautions to avoid a cookbook approach in diagnostic tasks and to examine the larger biopsychosocial context in which client symptoms occur.

The presence of the Novice Approach suggests that clinical experience may have some impact on treatment planning. These practitioners did not apply common cognitive schemas or clinical judgment strategies to integrate case input but appeared, instead, to simply recall descriptive information about the case in their written profiles. They took a long time to arrive at the correct diagnosis and, although expressing affiliation with the client's dilemma, were unaware of accurate base rates for ADHD. In fact, these participants performed similarly to advanced graduate students on the CTPS tasks (Falvey & Hebert, 1992). Although licensed and an average of eight years past their formal training, they demonstrated minimal clinical judgment (i.e., low correlations among all variables), and recorded significantly lower scores on the TP task than all other professionals in the sample. The relative lack of experience of clinicians in this cluster, compared to the larger sample, appeared central to their poor performance. Earlier research has concluded, however, that experience alone does not lead to increased cognitive complexity (Dawes, 1994; Garb, 1998). Given the importance of clinical judgment in the tasks represented by this research, it seems imperative to identify how this skill can be taught and learned during the early stages of a clinician's career.

The Efficient Approach may represent an optimal approach to these mental health tasks. These professionals were confident and knowledgeable about the disorder, and used both internal and external cognitive schemas to differentiate case input. They, more than other clinicians in the sample, actively sought suggestions from others (e.g., parents, teachers, guidance staff, family doctor) familiar with the case. Although they did not display affiliation for the client's unique circumstances, these clinicians were succinct in identifying the correct diagnosis and developing a viable treatment plan. Although their TP scores were not as high as the clinicians using the Mastery Approach, those using the Efficient Approach took considerably less time on this task and planned briefer treatment with a similar prognosis. This cluster of practitioners appears to demonstrate adequate competence and efficiency in comparison to the sample as a whole, and reflects the succinct problem-solving approach often required by large caseloads and managed care accountability standards.

This study contributes to the literature in several ways. First, the mental health professionals in this sample were full-time licensed practitioners representing several disciplines and both sexes, who were selected based on their specialization in psychotherapy with children in community and private practice settings. Averaging 13 years of clinical experience, participants reported that this simulated case was familiar (100%) and similar (96%) to cases they encounter in their clinical practice. This sample, thus, comprises a relatively experienced group of child clinicians familiar with the disorder represented, thereby distinguishing the present research from clinical judgment studies that frequently sample only trainees or new professionals (Garb, 1998; Hogarth, 1987; Turk & Salovey, 1988). Studying practicing clinicians is critical to advance the profession's understanding of clinical judgment in mental health practice.

Second, participant evaluations provided a degree of face and content validity in asserting that this methodology offers valid observations of cognitive processes that mediate between client data (i.e., input) and their associated outcomes (i.e., predictions). This link has been lacking in many clinical judgment studies (Strohmer & Leierer, 2000), and is central to the understanding of the cognitive processes of practitioners in the field.

Finally, an important finding of this study was that all three disciplines and most clinician demographics were represented in each of the four clusters. This result challenges the assumption of uniformity within, and differences between, the mental health disciplines. Training, license, gender, work setting, and clinical orientation were not found to be relevant to the outcomes of these case evaluation tasks. Although small sample size precludes definitive conclusions, this finding supports other evidence that intuitively important professional and personal variables are not, in fact, germane to clinical judgment in standard mental health tasks (Falvey, 2001; Garb, 1998; Turk & Salovey, 1988). What seems evident from the empirical literature is that, irrespective of the above demographics, mental health professionals appear to be quite diverse in their cognitive approach to case review and treatment planning (Suedfeldt et al., 1992). The current study further supports that conclusion. ]

Limitations of the Study

Given the qualitative strategy and small sample size used in this research, several limitations must be considered in any interpretation of the findings. First, clinicians who volunteer for a qualitative study may not be representative of their larger demographic cohort. Also, while the age and gender distribution of this sample appear to be largely representative of mid-career mental health practitioners, their lack of cultural diversity, although reflecting the New England community from which they were recruited, severely limits generalizability of these findings to the larger professional community.

A second limitation concerns the thinking aloud strategy that was used. While cueing was minimal, prompts to verbalize one's cognitive process may alter how one approaches judgment tasks. Some mental health professionals are not adept at verbalizing their thought processes (Faust, 1986; Garb, 1998). Alternately, some participants may have been influenced by the presence of the interviewer during the case review process. As with all studies using standardized simulations, this study provides an analog of decision-making processes, which may differ from those used in actual mental health counseling settings. Thus, the researchers cannot assume that what was articulated is a complete or accurate representation of these clinicians' thought processes. CTPS evaluations do, however, provide some evidence that the simulation and written tasks were perceived by participants as a fair representation of their clinical approach to case conceptualization and treatment planning. Finally, due to small cell sizes of this qualitative investigation, regression and cluster analyses must be interpreted primarily in terms of generating hypotheses for further study.

Implications for Mental Health Counselors

In the current study, patterns emerged that could guide future research and training. For example, longer time needed to identify a correct diagnosis, lack of knowledge of base rates, and the absence of cognitive strategies (e.g., Novice Approach) predicted poorer TP outcomes. Overreliance on diagnostic matching with little internal processing of case material (e.g., Template Approach) appears to limit TP considerations. Reliance upon an internal framework for evaluating case material, complex clinical judgment strategies, and awareness of base rates for a given disorder (e.g., Mastery Approach) were associated with the highest TP scores. Finally, active seeking of perspectives and guidance from other professionals regarding a case (e.g., Efficient Approach) appears to condense the process of case review and treatment planning but with some cost in terms of thoroughness. Such patterns could guide educators and supervisors to infuse graduate curricula and inservice training modules with specific strategies to enhance diagnostic and treatment planning skills. Recommendations include the use of case simulations, training in discrete cognitive skills and strategies associated with CC and TP, and the identification and modification of characteristic cognitive styles used by clinicians.

Use of Case Simulations. Given the growing need for evidence-based training, simulations offer a standardized format for teaching case conceptualization and treatment planning skills to graduate students in mental health counseling. Simulations also provide a viable, expert normed assessment tool to improve clinical skills in planning treatment for clients presenting with complex diagnostic and treatment issues (Falvey, 2001; Schwartz & Griffin, 1986). Medical, veterinary, and business schools commonly use standard simulations in training, but to date they are infrequently found in mental health training curricula. Focused practice using simulations has potential to contribute to the pre-service and inservice assessment of case conceptualization and treatment planning. For example, using clinical simulations like the CTPS as practice cases, counselors could (a) receive focused, expert normed feedback on their approach to these tasks; (b) identify their own characteristic problem-solving styles and clinical judgment strategies; and (c) observe how using specific cognitive strategies either enhances or limits performance on these tasks.

Teaching discrete CC and TP skills. Findings of this study, consistent with prior research, suggest that treatment planning scores differentiate performance across clinicians more clearly than do case conceptualization scores. Among 17 variables that addressed cognitive focus during the case reviews, 2 variables (i.e., Client Strengths, Symptoms Present in the Home) accounted for 33% of variance in TP scores, with no predictors of CC scores. Similarly, among 13 variables that addressed problem-solving styles, 2 variables (i.e., Synthesizes Input into Internal Framework, Relies on Own Knowledge of Disorder) accounted for 23% of TP score variance, with no predictors of CC scores. Among the 13 variables that addressed clinical judgment strategies, 2 variables (i.e., Verbalizes Awareness of Thought Process, Considers Client Strengths) accounted for 34% of CC score variance, with no predictors of TP scores. Several other variables that emerged from the cluster analysis suggest that considering at least 3 alternative DSM diagnoses, being aware of base rates of disorders, and avoidance of heuristic shortcuts were associated with higher TP scores. These findings provide some guidance to educators and supervisors to emphasize training in these discrete cognitive skills and strategies to improve counselor performance on treatment planning tasks.

Identifying Cognitive Styles. Do the clusters identified in this research represent state or trait aspects of clinician performance? Given that all respondents in this sample reported familiarity with this diagnosis and its similarity to their own cases, one could speculate that their performance reflects a stable approach to clinical problem-solving. If so, it may be useful in training mental health counselors to identify their characteristic problem-solving approach (i.e., cluster), and learn new strategies to counter problematic aspects of their problem-solving or clinical judgment to improve their skills in case conceptualization and treatment planning. Current research using the CTPS cases is focusing on the development of an interactive CD-ROM training program to identify and enhance these decision-making processes. Although designed for pre-service training, such a program could also afford a practical strategy to enhance inservice mental health practice.

CONCLUSION

Several conclusions emerge from this intensive study of problem-solving and clinical judgment. First, it is demonstrated that problem-solving styles and clinical judgment strategies can be identified that meaningfully distinguish performance in treatment planning, but not case conceptualization, among experienced mental health professionals. Second, each of three mental health disciplines sampled in the study were represented in all four resulting TP clusters, suggesting that professional demographics (i.e., degree, discipline) are not a valid indicator of skill on these judgment tasks. Third, specific variables demonstrated significant predictive value in performance on the TP task, thus providing support for skills-based training in these areas. Finally, participants rated all aspects of the research instrument as highly positive and representing a moderate degree of difficulty. The format and structure of this standardized simulation thus appear adequate for evaluating performance on these critical mental health assessment tasks.

Mental health counselors and professionals from related disciplines make important assessment and treatment planning decisions for every client they serve. Clarifying the cognitive processes and judgment strategies that influence those decisions both enhances our skills and provides accountability for our professional work. Understanding these factors has the potential to ground clinical practice in sound empirical research that will inform training and practice in relevant ways.

Author note: This research was supported by grants from the American Mental Health Counselors Association and the University of New Hampshire awarded to the senior author.

REFERENCES

American Psychiatric Association. (2000). Diagnostic and Statistical Manual--IV-TR. Alexandria, VA: Author.

Anderson, J. (1996). ACT: A simple theory of complex cognition. American Psychologist, 51, 355-365.

Arkes, H. R. (1991). Costs and benefits of judgment errors: Implications for debiasing. Psychological Bulletin, 110, 486-498.

Benbenishty, R., & Treistman, R. (1998). The development and evaluation of a hybrid decision support system for clinical decision making: The case of discharge from the military. Social Work Research, 22, 195-205.

Butcher, J. E., & Schofield, M. E. (1984). The use of a standardized simulation and process-tracing for studying clinical problem-solving competence. Counselor Education and Supervision, 24, 70-84.

Chessare, J. B., & Lieu, T. (1998). Teaching clinical decision-making to pediatric residents in an era of managed care. Pediatrics, 101(4), 762-767.

Dawes, R. M. (1994). House of cards: Psychology and psychotherapy built on myth. New York: Free Press.

Falvey, J. E. (2001). Clinical judgment in case conceptualization and treatment planning across mental health disciplines. Journal of Counseling and Development, 79, 292-303.

Falvey, J. E. (1994). [Clinical Treatment Planning Simulation: ADHD]. Unpublished document, University of New Hampshire. Clinical Judgment 34

Falvey, J. E. (1992). From intake to intervention: Interdisciplinary perspectives on mental health treatment planning. Journal of Mental Health Counseling, 14, 471-489.

Falvey, J. E., & Hebert, D. J. (1992). Psychometric study of the Clinical Treatment Planning Simulations (CTPS) for assessing clinical judgment. Journal of Mental Health Counseling, 14, 490-507.

Faust, D. (1986). Research on human judgment and its application to clinical practice. Professional Psychology: Research and Practice, 17, 420-430.

Gambrill, E. (1990). Critical thinking in clinical practice. San Francisco: Jossey-Bass.

Garb, H. N. (1998). Studying the clinician: Judgment research and psychological assessment. Washington, D.C.: American Psychological Association.

Gauthier, K. J. (1997). CTPS Computer Scoring [Computer program]. Durham, NH.

Hill, C. L., & Ridley, C. R. (2001). Diagnostic decision making: Do counselors delay final judgments? Journal of Counseling and Development, 79, 98-104.

Hogarth, R. (1987). Judgement and choice (2nd ed.). New York: Wiley.

Kahneman, D., & Tversky, A. (Eds.) (1982). Judgment under uncertainty: Heuristics and biases. Cambridge, MA: Cambridge University Press.

Kelly, M. B. (1977). A review of the observational data-collection and reliability procedures reported in the Journal of Applied Behavioral Analysis. Journal of Applied Behavioral Analysis, 10, 97-101.

Kleinmuntz, B. (1990). Why we still use our heads instead of formulas: Toward an integrative approach. Psychological Bulletin, 107, 296-310.

Ladany, N., Marotta, S., & Muse-Burke, J. L. (2001). Counselor experience related to complexity of case conceptualization and supervision preference. Counselor Education and Supervision, 40, 203-213.

Mahoney, M. J. (1988). Rationalism and constructivism in clinical judgment. In D.C. Turk & P. Salovey (Eds.), Reasoning, inference and judgment in clinical psychology (pp. 155-181). New York: Free Press.

Moore, S. R., Smith, R. E., & Gonzalez, R. (1997). Personality and judgement heuristics: Contextual and individual difference interactions in social judgment. Personality & Social Psychology Bulletin, 23, 76-84.

Nisbett, R. E., & Ross, L. (1980). Human inference: Strategies and shortcomings of social judgment. Englewood Cliffs, NJ: Prentice Hall.

Prieto, L. R., & Scheel, K. R. (2002). Using case documentation to strengthen counselor trainees' case conceptualization skills. Journal of Counseling and Development, 80, 11-22.

Schwartz, S., & Griffin, T. (1986). Medical thinking: The psychology of medical judgment and decision making. New York: Springer-Verlag.

Scolari (1996). QSR NUD.IST (rev. 3.0.5d) [Computer software]. Thousand Oaks, CA: Sage Publications Software.

Spengler, P. M., & Strohmer, D. C. (1994). Clinical judgment biases: The moderating roles of counselor cognitive complexity and counselor client preferences. Journal of Counseling Psychology, 41, 8-17.

Strohmer, D. C., & Leierer, S. J. (2000). Modelling rehabilitation counselor clinical judgment. Rehabilitation Counseling Bulletin, 44, 3-10.

Suedfeldt, P., Tetlock, P. E., & Streufert, S. (1992). Conceptual/integrative complexity. In C. P. Smith & J. W. Atkinson (Eds.), Motivation and personality: Handbook of thematic content analysis (pp. 393-400). New York: Cambridge University Press.

Turk, D. C., & Salovey, P. (Eds.) (1988). Reasoning, inference, and judgment in clinical psychology. New York: Free Press.

Tutin, J. (1993). The persistence of initial beliefs in clinical judgment. Journal of Social and Clinical Psychology, 12, 319-335.

Yennie, H. (1997). Treatment planning and clinical management software. Behavioral Health Management, 17, 8-10.

Janet E. Falvey, Ph.D., is affiliated with the University of New Hampshire. Timothy E. Bray, Esq., is affiliated with Taylor, Duane, Barton & Gilman in Boston. David J. Hebert, Ph.D. is affiliated with the University of New Hampshire. E-mail: [e-mail].

Table 1
Cognitive Focus During Case Review

Branched Categories                   M      SD

CLIENT PRESENTATION
  Symptoms Present in Home           3.16   0.47
  Symptoms Present in School         3.00   0.50
  Peer Relationships                 2.28   0.54
  Presentation at Intake             1.64   0.91
  Client Strengths                   2.36   0.81
RELEVANT HISTORY
  Medical History                    2.60   0.71
  Developmental/Family History       2.68   0.69
  Academic/School History            2.08   0.91
  Precipitant of Current Referral    2.44   1.23
SOURCES OF INPUT
  Parent Input                       2.92   0.28
  Teacher Input                      2.76   0.44
  Counselor Input                    2.24   1.05
  Client Input                       1.88   1.05
INITIAL EVALUATION
  Prior Interventions                2.48   0.59
  Formal Academic Testing            1.88   1.20
  Risk Assessment                    1.88   0.60
  Consistency Across Perspectives    2.20   0.58

                                       Strong or
                                        Primary
Branched Categories                      Focus

                                      N      %
CLIENT PRESENTATION
  Symptoms Present in Home            24     96
  Symptoms Present in School          22     88
  Peer Relationships                  8      32
  Presentation at Intake              3      12
  Client Strengths                    12     48
RELEVANT HISTORY
  Medical History                     14     56
  Developmental/Family History        14     56
  Academic/School History             9      36
  Precipitant of Current Referral     14     56
SOURCES OF INPUT
  Parent Input                        23     92
  Teacher Input                       19     76
  Counselor Input                     12     48
  Client Input                        9      36
INITIAL EVALUATION
  Prior Interventions                 11     44
  Formal Academic Testing             9      36
  Risk Assessment                     3      12
  Consistency Across Perspectives     5      20

                                         Minimal
                                           or
Branched Categories                     No Focus

                                      N      %
CLIENT PRESENTATION
  Symptoms Present in Home            1      4
  Symptoms Present in School          0      0
  Peer Relationships                  1      4
  Presentation at Intake              14     56
  Client Strengths                    4      16
RELEVANT HISTORY
  Medical History                     1      4
  Developmental/Family History        0      0
  Academic/School History             5      20
  Precipitant of Current Referral     6      24
SOURCES OF INPUT
  Parent Input                        0      0
  Teacher Input                       0      0
  Counselor Input                     4      16
  Client Input                        9      36
INITIAL EVALUATION
  Prior Interventions                 0      0
  Formal Academic Testing             11     44
  Risk Assessment                     6      24
  Consistency Across Perspectives     1      4

Table 2
Cognitive Schemas and Pooled Within-Groups Correlations Between
Canonical Discriminant Functions and Problem-Solving Styles

                                                      Not at
                                      Highly           all
                                  Characteristic  Characteristics

Cognitive Schemas      M      SD     N      %      N      %

Relies on
own knowledge
of this
disorder              2.6    0.57    15     60     0      0

Synthesizes
input into
internal
framework             2.4    0.76    14     56     4      16

Emphasizes
client based
treatment             1.6    1.00    6      24     14     56

Emphasizes
diagnosis based
treatment             2.0    0.84    7      28     4      16

Specifies
need for formal
evaluation            2.5    0.77    16     64     2      8

Seeks support
for initial
hypotheses            2.5    0.65    14     56     2      8

Seeks to
disconfirm
initial
hypotheses            2.6    0.65    17     68     2      8

Reviews all
available
information           2.4    0.76    13     52     2      8

Seeks direction/
assessment
from others           2.4    0.65    13     52     2      8

Anticipates
interventions
during
case review           2.3    0.89    13     52     5      20

Considers
hypotheses and
their
rationales            2.3    0.68    10     40     3      12

Expresses
affiliation
for client's
dilemma               1.5    1.20    7      28     11     44

Emphasizes
collaborative
treatment             2.6    0.49    16     64     0      0

Cognitive Schemas     Integration   Differentiation   Affiliation

Relies on
own knowledge of
this disorder             .28

Synthesizes
input into internal
framework                 .24

Emphasizes
client based
treatment                 .21

Emphasizes
diagnosis based
treatment                -.17

Specifies
need for formal
evaluation                .04

Seeks support
for initial
hypotheses                               -.58

Seeks to
disconfirm
initial
hypotheses                               -.43

Reviews all
available
information                               .37

Seeks direction/
assessment
from others                               .30

Anticipates
interventions
during
case review                              -.23

Considers
hypotheses and
their
rationales                               -.13

Expresses
affiliation
for client's
dilemma                                                   53

Emphasizes
collaborative
treatment                                                 35

* Discriminant function coefficients. TP cluster membership
was correctly predicted by these discriminant functions in
96% of the cases.

Table 3
Clinical Judgment Variables and Pooled Within-Groups
Correlations Between Discriminant Functions and
Clinical Judgment Strategies

                                     Impact on Case Review
Cognitive
Schemas                M    SD     High Impact   High Impact

                                     N      %      N      %

Demonstrates
cognitive
complexity            2.3   0.74     11     44      4     16

Confirmatory
bias                  1.7   0.69      3     12     11     44

Suspends
judgment
during case
review                2.3   0.69     11     44      3     12

Uses                  2.5   0.65     14     56      2      8
standardized
test norms

Primacy effect        1.4   0.65      2      8     17     68

Verbalizes
awareness of
thought process       2.2   0.72     10     40      4     16

Considers
client strengths      2.0   0.84      9     36      8     32

Seeks objective
assessments           2.6   0.56     16     64      1      4

Uses causal
reasoning model       1.8   0.78      5     20     11     44

Uses DSM
diagnostic
criteria              1.5   0.71      3     12     16     64

Representativeness
heuristic             2.0   0.65      5     20      5     20

Availability
heuristic             1.8   0.69      4     16      8     32

Considers
base rates of
disorders             1.1   0.44      4     16     20     80

                          Clinical Judgment Strategies *
Cognitive
Schemas               Minimal     Complex     Heuristic

Demonstrates
cognitive
complexity                  .06         .28

Confirmatory
bias                        .03        -.25

Suspends
judgment
during case
review                      .06         .19

Uses                        .14         .17
standardized
test norms

Primacy effect              .11        -.16

Verbalizes
awareness of
thought process             .02                    -.39

Considers
client strengths           -.05                    -.32

Seeks objective
assessments                 .14                    -.25

Uses causal
reasoning model             .06                    -.24

Uses DSM
diagnostic
criteria                    .14                     .23

Representativeness
heuristic                  -.04                     .18

Availability
heuristic                  -.02                     .16

Considers
base rates of
disorders                  -.03                     .12

* Discriminant function coefficients. TP cluster
membership was correctly predicted by these
discriminant functions in 100% of the cases.

* Note: Canonical discriminant functions
evaluated at group means

Table 4
Discriminant Analyses of Problem-Solving Styles
and Clinical Judgment Strategies in CTPS Treatment
Planning

Discriminant           Eigen     Canonical      % of     Wilk's
Function               value    Correlation   Variance   lambda

Problem-Solving
Style
  Integration           5.54         .88        65.48      .060
  Differentiation       1.14         .73        21.08      .271
  Affiliation           0.73         .65        13.44      .580

Clinical Judgment
Strategy
  Minimal Strategy      8.84         .95        73.23      .015
  Complex Strategy      2.12         .82        17.52      .152
  Heuristic Strategy    1.12         .73         9.25      .472

                                                          Cluster
                                                          Means *
                        [chi
                       square]       df           p         1
Problem-Solving
Style
  Integration           53.67       0.39        0.049     -1.54
  Differentiation       20.23       0.24        0.683     -0.15
  Affiliation           8.46        0.11        0.672      0.76

Clinical Judgment
Strategy
  Minimal Strategy      64.68       0.39        0.006      3.11
  Complex Strategy      29.24       0.24        0.211      0.11
  Heuristic Strategy    11.62       0.11        0.393      0.67

                               Cluster Means *

                          2           3           4
Problem-Solving
Style
  Integration           -1.17       3.02         0.49
  Differentiation       -1.18      -0.83         1.45
  Affiliation           -1.43       0.35        -0.42

Clinical Judgment
Strategy
  Minimal Strategy      -4.71      -1.38        -0.32
  Complex Strategy      -0.50       2.33        -1.52
  Heuristic Strategy     1.41      -0.81        -1.09