Improving physical therapy students' clinical problem-solving skills: an analytical questioning model.

Improving Physical Therapy Students' Clinical Problem-Solving Skills: An Analytical Questioning Model A student's problem-solving ability is very often a concern among educators and clinicians. [1] With the current knowledge level required in physical therapy, giving students the didactic knowledge necessary to function as a physical therapist often precludes extensive teaching at the application level. The students' ability to shift the learning process from mere accumulation of facts to creative and critical analysis of those facts, however, is essential for adequate evaluation and treatment of our patients and for growth of our profession. Clinical Instructors of the University of Kentucky's physical therapy program expressed their concern regarding students' clinical problem-solving ability when assessing patient problems and designing treatment programs. In addition, Kentucky recently joined the growing number of states having direct access to patient care through Kentucky Law KRS327.010. These two factors prompted the design of a model to improve the problem-solving abilities of our students.

In reviewing the literature, we found many studies that addressed creative ways to teach problem-solving skills but none that were objectively evaluated. Because time is a premium in physical therapy education, we wanted not only to add a problem-solving model to the curriculum but also to measure any improvement in the problem-solving ability of our students resultant from our model. If improvement occurred, we would know we had developed an effective model. Conversely, if no improvement occurred, we would modify our model.

The purpose of the study was to evaluate our problem-solving model to determine whether it could assist first-year physical therapy students in developing more effective problem-solving skills. The model was evaluated by comparing the results of the pretest-posttest performance on the Watson-Glaser Critical Thinking Appraisal (CTA) [2] and posttest feedback gathered through a questionnaire. The null hypothesis was that no difference would exist between control and experimental groups' performance on the CTA.

Problem-solving Models in the

Literature

Day states there is "an assumption that the ability to solve problems and to analyze new situations is paramount to good performance by a physical therapist." [3] To this end, a collection of problem-solving strategies has been designed for use in physical therapy curricula and clinics. [1,4-8]

May and Newman describe problem solving as "an internal and sequential process that includes cognitive, affective, and psychomotor behaviors." [1] They identify the following sequence of activities in their problem-solving model:

1. Problem recognition

2. Problem definition

3. Problem analysis

4. Data management

5. Solution development

6. Solution implementation

7. Outcome evaluation

Olsen [5] and Day [5] designed problem-solving models using a systems approach that requires each step to be analyzed in relation to other steps. Olsen states this approach "allows an instructor or clinical supervisor to monitor the student's problem-solving skills and facilitates feedback and exchange of information" using a structured sequence of analytical assessments and decisions. [5] Other problem-solving models use case histories to simulate actual clinical experience. [7,8,10] Our model was developed using feedback from local CIs and information gained from the studies mentioned previously. In designing our model, problem solving was defined as the nine behaviors identified in the Blue MACS (Mastery and Assessment of Clinical Skill) skill #42. [11]

Assessment of Problem-solving

Models

Problem-solving models generally are assessed by questionnaires and feedback from participants, a more subjective than objective assessment mode. Burnett et al developed a problem-solving model and concluded that

although no statistical data were collected to demonstrate that the problem-solving skills of the students improved by using this educational model instead of a more traditional model of clinical experience, students and clinicians reported the experiences were worthwhile. [8]

The authors of other problem-solving models stated similar conclusions. They found the models to be useful and effective [8,9] and cited positive feedback from students and clinical faculty. [5,7]

We hypothesized that the CTA would be an appropriate objective instrument to assess the effectiveness of our problem-solving model in this study. One use of the CTA postulated by its developers is "to measure gains in critical thinking abilities resulting from instructional programs in schools, colleges and business and industrial settings." [2] Studies have indicated that CTA reliability ranges from .85 to .87. [2] According to AJ Threlkeld (personal communication, November 1988), preliminary studies on validity at the University of Kentucky indicate a positive correlation between CTA scores of physical therapy students and grades students have earned in required physical therapy clinical sciences courses. One of the CTA's primary uses, therefore, appeared to be compatible with our intent.

The University of Kentucky's Physical Therapy Division has used the CTA for eight years as one of several preadmission screening tools for applicants to the physical therapy program. The CTA was selected as an instrument to predict students' critical thinking and problem-solving abilities, and the CTA score currently represents 10% of the total criteria used to determine eligibility for admission.

All students accepted into the physical therapy program had CTA scores on file, and these scores were used as baseline data for the students in the study. We believed that the CTA data would provide objective, standardized assessment of a problem-solving model that other models in physical therapy have lacked.

Method

Subjects

This study was submitted to the University of Kentucky's Institutional Review Board and qualified for exempt status. Subjects were the 31 first-year physical therapy students enrolled in Clinical Clerkship I, a four-week clerkship beginning in the spring semester. The class consisted of 30 female students and 1 male student ranging in age from 20 to 36 years. Prior to Clinical clerkship I, clinical experience for all subjects had been limited to preprofessional volunteer types of activities.

Procedure

In the University of Kentucky's physical therapy program, all first-year students are enrolled in Clinical Clerkship I. For this study, the class was divided into a Control Group (n = 15) and an Experimental Group (n = 16). The group assignments were based on a pre-existing random division of the class. Before attending the clinical clerkship, all students attended five lectures on various aspects of physical therapy. During the four-week clerkship period, all students attended one of seven general clinics for one-half day per week. In addition, students in the Control Group were required to observe 3 of 10 specialty clinics, and students in the Experimental Group were required to observe 2 of the 10 specialty clinics. Students in both groups were allowed to choose the specialty clinics.

Students in the Control Group generated an individual case study on a patient they had evaluated and treated according to an assigned format. These case studies were not discussed with the CIs, and no preclerkship or postclerkship meetings were scheduled.

Students in the Experimental Group met prior to the clerkship and were familiarized with the analytical questioning sequence that accompanied each of the five prepared case studies they would discuss with their CIs during the clerkship. They also generated an individual case study on a patient they had evaluated and treated. These additional case studies were discussed with the individual CIs and then with the other Experimental Group subjects during a postclerkship meeting. The total experiential span was six weeks.

The prepared case studies were designed for the Experimental Group and reflected typical patients seen in clinic. The case studies included 1) an inpatient with phlebitis, 2) a pediatric inpatient with a fractured fibula, 3) an outpatient witha sports-related knee injury, 4) an inpatient with a diabetic ulcer, and 5) a home care patient with a pinned fractured hip.

Each prepared case study was modeled after the physical assessment protocol students are instructed to use. [12] The evaluation consists of the following sections: 1) a patient history; 2) a subjective evaluation; and 3) objective findings consisting of inspection, function, palpation, neurologic, and special test sections. These prepared case studies were purposefully vague to generate student questions about further information required.

At the conclusion of the clerkship, two assessment methods were used. The CTA again was administered to the students in both the Control and Experimental Groups. The preadmission CTA scores were compared with this second set of CTA scores to assess the effectiveness of the problem-solving model. Questionnaires were sent to the students in the Experimental Group and to their CIs, and the respondents' comments were reviewed.

The first questionnaire item rated the students' assessment of their own ability to use didactic information as the basis for clinical decisions. The remaining items were short-answer questions and focused on obtaining the respondents' perception of 1) the value of the questioning sequence and case studies, 2) the assessment of student-CI interaction resulting from case-study discussions, 3) the clinical applicability of the experiences, and 4) the ways in which problem-solving skills were affected. All respondents also were asked whether they recommended continued use of the model.

Implementation of the Model

Preclerkship. Using guidelines from existing problem-solving models, [1,4-9] we developed a questioning sequence to facilitate students progressing through any case study with the intent of developing uniform problem-solving thought patterns. Basic categories of the model included in the questioning sequence were 1) assessment, 2) problem identification, and 3) treatment planning. The questions were applied to each of the five prepared case studies developed for this study.

Answering questions about hypothetical patients in a step-by-step fashion would be useful, but not adequate, for developing effective problem-solving skills. To develop effective problem-solving skills in the clinic, the CIs' involvement was critical. This belief was based on May and Newman's statement, "The role of the instructor is that of facilitator, rather than 'teller.'"

Before the clerkships began, we met for a two-hour session with the CIs to discuss the purpose of the study and the CIs' role. The CIs reviewed the five case studies and the questioning sequence design to be used with the Experimental Group. The final analytical questioning sequence shown in Table 1 incorporated their suggested revisions.

The CIs also generated a list of possible answers to each of the questions for each case study. A compilation of their responses was distributed to the CIs prior to the clerkships.

Strategies to facilitate student involvement through questioning were reviewed with the CIs. [13] These strategies were used during the weekly feedback sessions when the CIs and students discussed the case studies. The therapists were encouraged to use the same questioning strategies with the students to facilitate student involvement with other patients seen in the clinic.

Clerkship. During the orientation meeting, students in the Experimental Group reviewed Case Study 1 and completed the prescribed analytical questioning sequence. They then received Case Study 2, which they completed prior to beginning their clerkship. Students reviewed Case Study 2 with their CIs during the first week's feedback session. Students received Case Study 3 during the second week of the clerkship and again discussed it with their CIs. Students continued with Case Studies 4 and 5 during the third and fourth clerkship weeks, respectively.

During the third week of the clerkship, students wrote their individual case studies that were to be discussed with their CIs during the final week of the clerkship. Students also exchanged and discussed their case studies with another member of the Experimental Group group during the postclerkship meeting.

Data Analysis

After completion of all Clinical Clerkship I requirements, all students took the posttest CTA. The scores from the posttest CTA were compared with the preadmission CTA scores and were analyzed using a two-way analysis of variance (ANOVA). One Control Group subject's posttest CTA score was not included in the data analysis because a pretest CTA score was not available. Responses from the questionnaires also were reviewed.

Results

The three outlier CTA scores in the Experimental Group were considered in the data analysis. A Bartlett test for homogeneity of variance was performed to test the possibility that the three outlier scores on the posttest CTA in the Experimental Group skewed the results. [14] The range of change in scores for the Control Group was from -6 to +9, whereas the range of change for the Experimental Group was from -17 to +6, with outlier changes of -17, -15, and -15. With the outlier scores included, the variances were heterogeneous (X.sup.2 = 8.73, critical value = 5.02, p [is less than] .05). With the outlier scores removed, the range was from -12 to +6, and the Bartlett test showed the variances to be homogeneous (X.sup.2 = 1.34, critical value = 5.02, p [is less than] .05). The three Experimental Group outlier scores, therefore, were omitted from the data analysis.

A two-way ANOVA was performed to determine whether there were differences in the overall performances of the two groups and whether there were changes in performance from pretest to posttest administration of the CTA.

There was no significant difference either between the two groups (F[1,32] = 1.05, p [is greater than] .05) or in change in performance (F[1,7] = 0.81, p [is greater than] .05). The interaction between the two groups and performance also was not significant (F[1,43] = 4.92, p [is greater than] .05). the means and standard deviations for both groups on the pretest and posttest administration of the CTA appear in Table 2.

Participant feedback was gathered from the questionnaires returned by 7 Experimental Group subjects and by 7 of the 8 participating CIs. All respondents indicated they often or routinely used didactic information to make clinical decisions. The respondents commented that the questioning sequence was of value in several ways. Two sample comments were: "After I started answering the questions in sequence, solutions to the problems followed naturally" and "I feel I can better think things/processes through in a more logical, orderly fashion."

Students' assessment of the feedback sessions with the CIs was very positive. Students stated: "I thought that this was a very valuable aspect of the course. The extra time spent on the discussion helped me get a perspective on how clinicians go about solving problems and served as examples that I will probably follow throughout my career" and "This was excellent. Not only did the case for the week get discussed, but other problems and possibilities were talked about. This made me feel more comfortable with the CI in other events."

Finally, students responded that the experience was clinically applicable. One respondent stated, "This experience will give me more confidence in deciding how a patient should be treated." The continued use of the model was recommended by the students in the Experimental Group.

Comments from the CIs' questionnaires indicated that they thought the case

studies and questioning sequence were of value. For example, one CI stated the model "gave a focus to work with students on problem-solving, and gave students more insight on how they can be involved with patients in their first clinical exposure."

Clinical Instructors also responded that the experience was applicable clinically and had carry-over to other areas. One CI stated the model "helps students with what content should be in their note writing, (and) how to use objective measurements." Another CI commented, "Good system, (the) questioning sequence helped students design treatment programs with other patients they saw in the clinic." All CIs recommended that the model continue to be used with students in Clinical Clerkship I and further recommended the model to be used with second-year students.

Discussion

The CTA results did not allow for rejection of the null hypothesis. The problem-solving model, however, was found to be useful, effective, and generally worthwhile to both the students in the Experimental Group and to the participating CIs, as indicated by their questionnaire responses. Although the effectiveness of the model was not validated by the CTA, several explanations deserve discussion. As separate or combined factors, the study could have been affected by the assessment tool, student motivation, and the design and implementation of the problem-solving model.

The choice administration of the assessment tool appear to have affected the results of the study. The baseline CTA scores had a mean of 83, which is relatively high. This mean corresponds to a percentile rank of 97% to 99%. [2] With initial scores this high, it is more difficult to show improvement, and there may be a tendency for regression toward the mean.

The application of skills tested by the CTA may not be as compatible with the specific problem-solving content that was intended in the model as it appeared to be. In the opinion of the Assistant Director of the University of Kentucky's Counseling and Testing Center, the CTA more appropriately tests the ability of students to work with abstractions in a reading situation (Peg Taylor, PhD; personal communication; July 1987). The questions for each case study required concrete reasoning, as opposed to the CTA's abstract reasoning mode. In addition, the CTA is intended to be an untimed test that usually takes at least an hour to complete. The preadmission CTA was untimed, but because of a class-schedule conflict, the postclerkship CTA was limited to 45 minutes.

Student motivation is an intangible, but influential, factor. The first time the CTA was administered to the students they were highly motivated to gain acceptance to the physical therapy program. That high level of motivation may not have been present during the postclerkship administration of the CTA. Additional required meeting times for the students in the Experimental Group also may have been a negative influence. The clerkship is a pass-fail course, and students knew that the outcome of the second CTA examination would not affect their clerkship grade.

Another uncontrolled factor was the CIs' participation. This was the first time the CIs had used the model; therefore, it is possible that their lack of experience with the model affected their ability to implement it. In several instances, because of work schedules and illness, the CI with whom the student met weekly changed. Some students in the Experimental Group, therefore, met with CIs who had not attended the two-hour preclerkship orientation session, had not been exposed to the purpose of the study, and had not been trained in the problem-solving analytical questioning sequence and student-involvement techniques. The study's design thus was disrupted, because the role of the CI was vital to the success of the model. In the implementation of the model, the variability of the CIs needed to be controlled to a greater degree than that of the students. The possibility also exists that our model was not effective.

At the time the junior physical therapy students began their clerkships, they had not been introduced to problem solving in the curriculum, and the four-week clerkship is the first time they are in the clinic. One-half day once a week is a relatively short time for students to adapt to the clinic and to acquire, assimilate, and use new information. Although the clerkship time is relatively limited, it is a very important part of their clinical education.

Attention to the development of problem-solving skills is essential early in students' clinical experience because initial thought patterns are established that become the foundation for entry-level practice. The quality of physical therapy services provided by the students during their clinical rotations can be affected positively if appropriate instructional methods are used that focus on problem-solving development.

Junior physical therapy students have no base of experience upon which to draw; they obviously lack professional experience and practice. Sanders and Ruvolo designed Mock Clinics primarily to develop students' awareness and insight into their own clinical knowledge, behaviors, and interactions with clients. [15] Although the Mock Clinics did not focus on developing problem-solving skills, the mock patient treatment sessions demonstrate one method to help students create a needed frame of reference for generating hypotheses and for enhancing the development of clinical problem-solving skills. The analytical questioning sequence applied to the six case studies with the Experimental Group represents our approach to simulating professional experience and practice.

Bruner states that effective instruction allows the learner to translate information into problem-solving efforts and that the goal of instruction should be to facilitate students to become self-propelled thinkers. [16h Elstein and associates studied how the translation of information occurred in medical practice and described the clinical reasoning process used by physicians as cue acquisition, hypothesis generation, cue interpretation, and hypothesis evaluation. [17] Payton determined that experienced physical therapists used a clinical problem-solving sequence comparable to the physicians' sequence. [18] In both the Elstein et al [17h and Payton [18] studies, skilled practitioners began generating tentative hypotheses very early during patient contacts, hypotheses based on their experience prior to acquiring all the needed data from histories and physical examinations.

Elstein et al state that

investigations of problem solving in chess, in logic, and in medicine are converging on the same conclusion. The differences between experts and weaker problem solvers are more to be found in the repertory of their experiences. [17]

An important component in the early clinical rotations is a structured problem-solving sequence of questions for data collection that initiates the creation of such a repertory. Successful instruction in problem-solving skills at the first-year level can provide a sound base for extended problem-solving development at the second-year level and at the professional entry level and can provide improved physical therapy services at all levels.

Positive participant feedback should not be discounted. The basis for evaluating other models to improve problem-solving skills have been participant feedback. [5,7] The feedback on our model was unanimously positive from those students who completed the questionnaire and those who did not.

Further research is needed in the area of problem-solving skill development in physical therapy students. The CTA and other standardized tools may be adjuncts to objectively validate many of the existing problem-solving models that have been assessed subjectively. In addition, studies of whether the CTA is a valid tool to assess changes in physical therapy students' critical thinking abilities are necessary.

Conclusions

Objective improvement in students' problem-solving skills as measured by the CTA was not apparent in this study. If the CTA is a valid instrument for measuring problem-solving skills, then the effectiveness of our model and its implementation may be questioned. Our participants strongly recommended that the model continue to be developed and used with our physical therapy students. In line with the recommendations, our model is being revised, but its degree of effectiveness will continue to be contingent on the CI's skill in its implementation.

If the CTA is not a valid instrument, other appropriate objective standardized assessment tools must be identified or developed to measure progress in physical therapy students' problem-solving abilities and skills as a result of clinical education. Valid assessment tools will ensure that we are providing our students with effective skills and future patients with effective evaluation and treatment.

Acknowledgment 

We acknowledge the clinicians who gave of their time, experience, and expertise.

References

1 May BJ, Newman J: Developing competence in problem solving: A behavioral model. Phys Ther 60:1140-1145, 1980

2 Watson G, Glaser EM: Critical Thinking Appraisal Manual. Cleveland, OH, Psychological Corporation, 1980

3 Day JA: Graduate Record Examination analytical scores as predictors of academic success in four entry-level master's degree physical therapy programs. Phys Ther 66:1555-1562, 1986

4 Dunkle SE: Developing a Problem-Solving Approach to Teaching Physical Therapy Skills. Doctoral Dissertation. Nova University, Ft Lauderdale, FL, 1982

5 Olsen SL: Teaching treatment planning: A problem-solving model. Phys Ther 63:526-529, 1983

6 Barr J: A problem solving curriculum design in physical therapy. Phys Ther 57:262-270, 1977

7 van der Sijde PC, Sellink WJL, Wurms RJ: Developing audiovisuals for problem solving in physical therapy education. Phys Ther 67:554-557, 1987

8 Burnett CN, Mahoney PJ, Chidley MJ, et al: Problem-solving approach to clinical education. Phys Ther 66:1730-1733, 1986

9 Day DJ: A systems diagram for teaching treatment planning. Am J Occup Ther 27:239-243, 1973

10 Henry JN: Identifying problems in clinical problem solving: Perceptions and interventions with nonproblem-solving clinical behaviors. Phys Ther 65:1071-1074, 1985

11 Mastery and Assessment of Clinical Skill. Developed by the Texas Consortium for Physical Therapy Clinical Education, Galveston, TX, 1980

12 Nitz AJ: Course Syllabus PT 807/846. Department of Physical Therapy, College of Allied Health Professions, University of Kentucky, Lexington, KY, 1987, p 6

13 Bowling B: Questioning: The Mechanics and Dynamics, TIPS (Teaching Improvement Project System). Department of Allied Health Education and Research, College of Allied Health Professions, University of Kentucky, Lexington, KY, 1978

14 Rosner B: Fundamentals of Biostatistics. Boston, MA, PWS-KENT Publishing Co, 1982, pp 435-438

15 Sanders BR, Rubolo JF: Mock Clinic: An approach to clinical education. Phys Ther 61:1163-1167, 1981

16 Bruner JS: Toward a Theory of Instruction. Cambridge, MA, Harvard University Press, 1966, p 53

17 Elstein AS, Shulman LS, Sprafka SA, et al: Medical Problem-Solving: An Analysis of Clinical Reasoning. Cambridge, MA, Harvard University Press, 1978, pp 273-302

18 Payton OD: Clinical reasoning process in physical therapy. Phys Ther 65:924-928, 1985

D Slaughter, MSEd, PT, is Director of Physical Therapy, University of Kentucky Sports Medicine Center, Medical Plaza, Rm D-103, Lexington, KY 40536 (USA). She was a student in the Curriculum and Instruction Clinical and College Teaching Program, University of Kentucky, when this study was completed in partial fulfillment of the requirements of her master's degree.

D Brown, MSEd, PT, is Assistant Professor and Academic Coordinator of Clinical Education, Physical Therapy Division, College of Allied Health Professions, University of Kentucky.

D Gardner, MEd, is Professor, Allied Health Education and Research Program, College of Allied Health Professions, University of Kentucky.

L Perritt, PhD, is a part-time assistant professor, Allied Health Education and Research Program, College of Allied Health Professions, University of Kentucky. Dr Perritt is also a licensed psychologist in private practice.

This article was submitted September 25, 1987; was with the authors for revision for 43 weeks; and was accepted February 10, 1989.