Effect of including a clinical example on the ability of physical therapists to apply information in a technical research report.

The ongoing acquisition of new knowledge is the cornerstone of effective clinical decision making.[1] Knowledge is the base upon which decision analysis proceeds, guiding questions that should be posed or interventions that should be used. Throughout the course of any decision tree, knowledge provides the standard for comparisons with patient data acquired during assessment and treatment.[2] Journal reports of research could be an important contemporary source of new knowledge for clinicians, yet few clinicians identify these research reports as their primary source of information for clinical decision making.[3] Rosen[4] suggests that the method used to communicate research is one of the major factors that influences the clinical application of research. The communication or failure in communication of the research report may provide a major barrier to effective application of research by clinicians. Sparling and Gallagher[5] criticize the format of the typical research article as the "least informative communication known to man." Research reports in journals often are described by clinicians as neither user-friendly nor user-ready.[6]

Several factors may deter the effect communication of information through the research report and the ability of clinicians to apply that information in clinical practice. Bohannon[7] reported that physical therapists he interviewed complained that research reports often were too technical and esoteric, contained complex statistical analyses that could not be understood, and did not seem to apply to clinical practice. Other therapists indicated that research reports often were too long and were organized poorly.[7]

Hunt[8] and Sekerak[9], have suggested that a major barrier to effective use research may be the inability of clinicians to apply research they have read about. Both Halpert[10] and Scholl[11] advocate that the clinical utility of research reports can be enhanced by authors, concerted efforts to include specific information regarding the clinical application of information in the report. This recommendation is consistent with the writings of learning theorists, who suggest that problem solving and the transfer of learning are fostered by activities that provide learning guidance and emphasize meaningful application of information.[12,13] Gagne and Driscoll[14] suggest that the retrieval of information and transfer of information to new problem-solving efforts are enhanced by providing specific examples of how the information can be used in different contexts. They also suggest that the lateral transfer or generalization of learning to new situations is promoted by presenting a variety of situations or problem contexts in which the learned information is applied through active practice by the learner.

Journal reports of research can be a resource or learning tool for new knowledge in the process of clinical decision making. The writings of Gagne and Driscoll[14] suggest that the inclusion of specific application examples in a research report should promote learning and the ability of the clinician to transfer new knowledge to the clinical decision-making process. Although learning theory may provide an intuitive and theoretical basis for this hypothesis, no research support for this position exists in the physical therapy literature.

The purpose of this study was to determine whether inclusion of an example in a technical research report would affect the application of information included in the report. The following specific research questions were addressed:

1. Does inclusion of an example of applying information in a research report affect the frequency with which physical therapists select correct mathematical models for predicting preinjury muscle performance?

2. Does inclusion of an example of applying information in a research report affect the frequency with which physical therapists derive correct-solutions to hypothetical clinical problems?

3. Does inclusion of an example of applying information in a research report influence future utilization of information in the report?

Method

Subjects

We obtained from Lumex Inc the names of 82 clinical facilities that used Cybex[R] isokinetic devices(*) and then telephoned the director of each facility to identify potential subjects for the study. We attempted to identify one physical therapist at each facility who was willing to participate in the study, and eventually identified 69 therapists who were willing to do so. These 69 therapists were randomly assigned to two groups: group 1 (n=35) and group 2 (n=34). All subjects reported that they routinely used Cybex[R] isokinetic equipment in their clinics to evaluate and treat patients with knee joint pathologies. Subjects also reported that they conducted routine testing of patients at regular intervals during the course of patients, rehabilitation.

Procedure

Each subject completed a demographic data questionnaire to indicate age, gender, academic degree for physical therapy entry-level program, years since graduation from a physical therapy entry-level program, other academic degrees obtained, the number of statistics courses taken, and the number of years since the last statistics course was taken. Subjects also indicated on the questionnaire whether they read the journal Physical Therapy regularly and were involved in research. Finally, each subject reported the number of patients with knee pathologies he or she treated per month, whether physical therapy students affiliated in his or her clinic, and the number of physical therapists working in his or her clinical facility. The previously described demographic variables were selected based on their relationship with research utilization in clinical decision making by physical therapists.[9]

Subjects in group 1 were asked to read a manuscript of a research report that appeared in Physical Therapy following completion of this study.[15] At the time of this study, the manuscript had been accepted for publication. Editorial staff at the journal granted permission for use of the manuscript in the study. The research report mailed to group 1 subjects included mathematical models that could be used to predict preinjury knee flexion and extension torque values produced on Cybex[R] isokinetic equipment. The mathematical models were based on multiple regression analyses, and their use involved manipulation of patients, demographic and anthropometric characteristics. The research report concluded with a section that provided an application example of how the models could be applied for a hypothetical patient problem. The example consisted of a demonstration of how values for predictor variables would be substituted in one prediction equation.

Group 1 subjects were asked to read the research report, solve clinical problems for two hypothetical patients (Appendix), and then mail their problem solutions and the demographic questionnaire to the investigators. Group 1 subjects were asked to select for each patient the appropriate prediction equation and then to use the equation they had selected to obtain a predicted torque value. We recorded for each subject the number of correct prediction models selected: 0, 1, or 2. We also recorded for each subject the number of correctly derived torque values: 0, 1, or 2. Subjects were credited with a correct torque value determination if they applied the selected prediction equation correctly, even if they had selected an incorrect prediction equation.

The previously described testing procedures were applied to subjects in group 2, with one exception, Subjects in group 2 received the same manuscript mailed to group 1 subjects, except the application example was omitted.

Subjects were asked to work independently to comprehend information in the research report they received and derive solutions to the hypothetical patient problems. Each subject was mailed a copy of the correct solutions to the clinical problems as soon as he or she returned the demographic questionnaire and the answers to the two hypothetical patient problems. We attempted to contact each subject by telephone 6 to 12 weeks after the subject had returned the questionnaire and their answers to the patient problems. Subjects were asked at that time whether they had used information in the manuscript to identify knee strength rehabilitation goals for their patients.

Data Analysis

The two groups were compared qualitatively with regard to subject characteristics acquired with the demographic data questionnaire. A chi-square analysis was used to determine whether inclusion of an example of applying information in a research report affected the frequency with which subjects were able to identify the correct knee torque prediction model. A second chi-square analysis was used to determine whether inclusion of an example of applying information in a research report affected the frequency with which subjects were able to compute predicted torque values correctly. Data were analyzed qualitatively to determine whether inclusion of an example of applying information in a research report influenced future utilization of information in the report.

Results

Twenty of the group 1 subjects (67%) and 21 of the group 2 subjects (70%) returned the demographic data questionnaire and solutions for the two hypothetical patient problems. Descriptive statistics for subject demographic data are presented in Table 1 for these 41 subjects. Additionally, 2 subjects in each group had earned master's degrees following completion of a baccalaureate entry-level degree in physical therapy.

[TABULAR DATA OMITTED]

Table 2 represents the contingency table used to determine whether inclusion of an example of applying information in a research report was related to the frequency with which subjects were able to identify the correct knee torque prediction model. The total number of selected prediction models was 40 for group 1 and 42 for group 2 (2 selections per subject). The value of the chi-square test statistic ([x.sup.2] = 0.16, df =1) indicated the frequency of selecting the correct knee torque prediction model was not related to the inclusion of the example of applying the information in the research report. Most prediction models were identified correctly (83% for group 1 and 86% for group 2), regardless of the manuscript version received by subjects.

Table 2. Contingency Table Results(a)
for Relationship Between Format of
Research Report(b) and Ability to Identify
Correct Prediction Models

        Correct      Correct
Group   Selections   Selections

1       33           7
2       36           6

(a) Values are frequency of occurrences.

(b) Group 1=subjects who read research report
that included a clinical application example;
group 2=subjects who read research report that
did not include a clinical application example.

Table 3 represents the contingency table used to determine whether inclusion of an example of applying information in a research report was related to the frequency with which subjects were able to compute predicted torque values correctly. The total number of computed torque values was 40 for group 1 and 42 for group 2 (2 computations per subject). The value of the chi-square test statistic ([x.sup.2] = 9.35, df = 1) indicated group 1 subjects were more likely to compute correct knee torque values (P.<.01) than group 2 subjects. Approximately 88%, of the torque computations were correct for the group 1 subjects who had read the research report that included the application example. The errors made by the group 1 subjects were use of incorrect units (n=2), failure to square the value for age (n=1), and coding gender incorrectly (n=2). Approximately 57% of the torque computations were correct for the group 2 subjects who read the research report without the application example. The 18 incorrect torque answers for the group 2 subjects involved the following 21 errors: coding gender incorrectly (n=1), expressing percentage of body fat in decimal form rather than as a percentage value (n=8), math computational errors (n=3), digits copied incorrectly (n=3), use of incorrect units of measurement (n=1), and alteration of the originally selected formula (n=1). The total number of errors for group 2 equaled 21 because 2 different types of errors were made for 3 of the solutions.

Table 3. Contingency Table Results(a)
for Relationship Between Format of
Research Report(b) and Ability to Derive
Predicted Torque Values Correctly

         Correct    Correct
Group    Answers    Answers

1        35          5
2        24         18

(a) Values are frequency of occurrences.

(b) Group 1=subjects who read research report
that included a clinical application example;
group 2=subjects who read research report that
did not include a clinical application example.

For the follow-up analysis for research question 3, we were able to contact 19 (10 for group 1 and 9 for group 2) of the 41 subjects who returned the demographic data questionnaire and solutions for the two hypothetical patient problems. Three subjects contacted from group 1 reported that they had used information in the manuscript to identify knee torque rehabilitation goals for their patients. None of the subjects contacted from group 2 reported having used the research for identification of rehabilitation goals.

Discussion

The two subject groups were similar with regard to demographic data (Tab.1), except for the number of patients with knee conditions treated per month. The dissimilarity between groups for this variable is attributable to the unusually large number of patients with knee conditions treated per month for four group 1 subjects: 55, 70, 75, and 80 patients per month.

The results indicate that inclusion of an example of applying information in a research report was not related to the frequency with which subjects were able to identify the correct knee torque prediction model and that most subjects were able to identify the correct knee torque prediction models. This result is not surprising given the nature of what was asked of subjects and the manner in which the prediction models were presented in the research reports. Selection of the correct prediction models required that subjects only recognize which model was appropriate and did not require that subjects apply the model they selected to the patient problems. In addition, each prediction model was presented in table format in both manuscripts, and the title of each table provided sufficient information that would enable correct selection of the appropriate model.

Group 1 subjects derived the correct answers to the clinical problems more frequently than group 2 subjects. This result supports the recommendations of Halpert[10] and Scholl[11] that the utility of research reports can be enhanced by inclusion of specific information regarding application of information included in the report. Even though most subjects in both groups were able to select the correct prediction model, the value of this ability for the group 2 subjects was limited because of their relative inability in the technical application of the model.

The relationship between exposure to an application example and the greater frequency with which the group 1 subjects derived correct solutions to the hypothetical problems also supports the theory of lateral transfer described by Gagne and Driscoll.[14] The example of applying information that was included in the manuscript read by the group 1 subjects involved only a demonstration of how values for predictor variables would be substituted in one prediction equation with no rationale for their inclusion. The rationale for inclusion of the predictor variables in each prediction equation was given in the body of the manuscript that was read by both subject groups. The inclusion of this basic demonstration, however, was related to a reduced frequency of technical application errors being made by group 1 relative to group 2.

The only attempt to control math computational skills (ie, addition, subtraction, and so forth) involved random assignment of subjects to the two groups. Group 1 subjects made 5 technical application errors and no mathematical computational errors. Group 2 subjects made 15 technical application errors (coding gender incorrectly, expressing percentage of body fat in decimal form rather than as a percentage value, use of incorrect units of measurement, and alteration of the originally selected formula) and 6 mathematical computational errors (math errors and incorrectly copied digits). Thirteen of the 15 technical application errors made by group 2 involved coding gender incorrectly and use of incorrect format for percentage of body fat values. Correct measurement and recording of both of these variables was discussed in the body of the manuscript read by both subject groups, but the inclusion of these variables in the prediction models was demonstrated only in the application example read by the group 1 subjects. Inclusion of application examples for research reports that involve technical information may assist the clinician in applying information if the report includes potentially ambiguous information (eg, correct format for expression of percentage of body fat) or if critical information is identified only once in the report (eg, correct coding scheme for gender). Gage and Berliner[16] and Mayer[17] support this contention by suggesting that an important reason for including models or examples is to direct the learner's attention to key details of the learning task.

We experienced appreciable difficulty contacting the subjects by telephone to determine the number of subjects who had applied information in the manuscript to identify knee torque rehabilitation goals for their patients. We did not continue to try to contact subjects following three unsuccessful telephone attempts. All subjects who indicated they had applied manuscript information in the clinic were from group 1. Insufficient data, however, were available to judge whether inclusion of a clinical application example in a research report affected the continued clinical application of information presented in the report. The relatively small number of subjects (3 of 19 contacted) who indicated they had applied information in the manuscript in their clinical practice is consistent with other data on utilization of research by physical therapists.[2,7] The reasons why a majority of these clinicians had not applied information in the manuscript in their clinical practice were not identified, but may have included the following: They did not understand the information, they did not believe the information, they did not know how to apply the information, they were not allowed to apply the information, or they did not find the information useful.[9]

Finally, we wanted to determine whether inclusion of an application example in a research report would facilitate application of information included in the report. Authors who choose to include examples of applying information in their research reports should be concerned with the possibility that, under certain circumstances, an application example might lead to either rigid or inappropriate application of research data. The term negative transfer has been used to describe such a scenario in which previous learning hinders new learning or problem solving. Application examples should provide the guidance to facilitate transfer of learning without being excessively simplistic or focused. Negative transfer can be avoided by attempts to teach flexibility in applying principles and by explicitly indicating instances in which principles may or may not apply.[16]

Clinical implications

The results of this study indicate that including an example of applying information in a technical research report may enhance the ability of clinicians to use the information in the report. These results may or may not be applicable to theoretical or less technical journal articles, or to the effects inclusion of application examples in these articles may have on clinical practice. Our research effort assessed a limited and technical skill that might be useful in the clinical decision-making process: applying prediction models to establish muscle performance rehabilitation goals. Even though both subject groups knew the correct approach to take in solving the hypothetical patient problems, the group 1 subjects who read the manuscript with the application example were able to complete the requested technical task accurately with greater frequency compared with the group 2 subjects.

Additional research may document the ability to generalize our results to other types of journal articles and other aspects of the clinical decision-making process. Such results would suggest that investigators should include examples of applying information in certain types of manuscripts; that editorial staffs for professional health care journals should require, when appropriate, application examples in submitted manuscripts; and that physical therapy educators should foster the development of this practice by requiring that student research reports include similar content.

References

[1] Wolf SL. Summation: identification of principles underlying clinical decisions. In: Wolf SL, ed. Clinical Decision Making in Physical Therapy, Philadelphia, Pa: FA Davis Co; 1986:379-384. [2] Ballin AJ, Breslin WH, Wierenga KS, Shepard KF. Research in physical therapy: philosophy, barriers to involvement, and use among California physical therapists. Phys Ther. 1980;60: 888-895. [3] Bohannon RW, LeVeau BF. Clinician's use of research findings: a review of literature with implications for physical therapists. Phys Ther. 1986;66:45-50. [4] Rosen A. Barriers to utilization of research by social work practitioners. J Soc Serv Res. 1983;6(3/4):1-5. [5] Sparling JJ, Gallagher JJ. The need to communicate. In: Sparling JJ, Gallagher JJ, eds. Research Direction for the 70's in Child Development, Chapel Hill, NC: Frank Porter Graham Child Development Center; 1971:25. [6] Goode CJ, Lovett MK, Hayes JE, Butcher IA. Use of research-based knowledge in clinical practice. J Nurs Adm. 1987;17(12):11-18. [7] Bohannon RW. Information Accessing Behavior of Physical Therapists. Raleigh, NC: North Carolina State University; 1988. Dissertation. [8] Hunt J. Indicators for nursing practice: the use of research findings. J Admin Nurs. 1981;6: 189-194. [9] Sekerak DK. Characteristics of Physical Therapists and Their Work Environments Which Foster the Application of Research in Clinical Practice. Chapel Hill, NC: The University of North Carolina at Chapel Hill: 1992; 16-62. Dissertation. [10] Halpert HP. Communications as a basic tool in promoting utilization of research findings. Community Ment Health J. 1966;2:231-236. [11] Scholl GT. Bridges from research to practice in education of visually handicapped people. J Vis Impairment Blindness. 1983;77:340-344. [12] Gagne RM, Briggs LJ. Principles of Instructional Design. New York, NY: Holt, Rinehart and Winston Inc; 1974:68-69. [13] Davis RH, Alexander LT, Yelon SL. Learning System Design: An Approach to the Improvement of Instruction. New York, NY: McGraw-Hill Inc; 1974-247-277. [14] Gagne RM, Driscoll MP. Essentials of Learning for Instruction. Englewood Cliffs, NJ: Prentice Hall; 1988;90-94. [15] Gross MT, McGrain P, Demilio N, Plyler L. Relationship between multiple predictor variables and normal knee torque production. Phys Ther. 1989;69:54-62. [16] Gage NL, Berliner DC. Educational Psychology. Boston, Mass: Houghton-Mifflin Co; 1991;306-325. [17] Mayer RE. Models for understanding. Rev Educ Res. 1989;59:43-64.