Use of The Blue MACS: Acceptance by Clinical Instructors and Self-Reports of Adherence.

A purpose of a clinical education evaluation instrument is to provide a valid and reliable measure of student performance to ensure that people entering the field will be competent.[1,2] The Texas Consortium for Physical Therapy Clinical Education, whose members are the physical therapy programs in Texas, began developing The Blue MACS in 1977 with extensive input from clinicians.[3] The Blue MACS has been revised since its introduction and is currently in its sixth edition.[4] In 1996, the fifth edition of The Blue MACS, which was in use at the time this study was conducted, was thought to be used in 38% of the physical therapist education programs in the United States (Lucy Gibbs, former Executive Director of the Texas Consortium for Physical Therapy Clinical Education; personal communication). The Blue MACS is intended to be competency based and criterion referenced.[3,4] It was designed to encompass a comprehensive list of the entry-level physical therapy skills that a student must master, each defined by specific criteria (key indicators) describing entry-level mastery of the skill. In The Blue MACS, entry level is defined as "the level of knowledge, skills, and professional behavior expected of a new graduate when entering the profession of physical therapy."[4(p5)]

The Blue MACS consists of a list of 38 skills and 12 additional "situational skills" that may not be available in all settings. Each skill has a list of 4 to 12 key indicators (see Fig. 1 for a sample skill and key indicators). Students rate their own performance on each key indicator before the clinical instructor (CI) rates the key indicators. This procedure provides a format for discussion of student performance based on similarities and differences in ratings. Key indicators are rated "+" for exceeds entry level, "[check]" for meets entry level, "NE" for needs experience to meet entry level, and "NI" for needs improvement to meet entry level. No rating for a key indicator means that the student has not been working on that key indicator. When all of the key indicators for a skill have been approved, the skill is approved. A skill or key indicator approved during a previous clinical education experience may be challenged by a CI if the student does not, in the opinion of the CI, meet the standard for entry-level practice for a later clinical education experience setting. A challenge requires the student to meet the demands made at the new setting for entry-level practice before the skill or key indicator can be reapproved. There is space on The Blue MACS form for comments on each skill and for a listing of modalities, exercise equipment, and evaluation techniques by diagnosis that the student has experienced. The Blue MACS is designed to be carried from one clinical education experience to the next, so that the CI is provided with a description of previous clinical experiences and remaining educational needs.

[Figure 1 ILLUSTRATION OMITTED]

In 1997, the Physical Therapist Clinical Performance Instrument (CPI) was introduced by the American Physical Therapy Association (APTA) for the purpose of evaluating the performance of students during their clinical education experiences.[5] The CPI differs from The Blue MACS in its approach to the evaluation of the clinical education experience. The Blue MACS requires the CI to approve very specific key indicators of behavior, whereas the CPI asks for ratings of 24 broader categories of behavior, with each category of behavior rated on a visual analog scale (VAS). The lowest limit is labeled "novice clinical performance," and the highest limit is labeled "entry-level performance." Sample behaviors are provided to suggest the type of behaviors to be considered for each of the broad categories of behavior, but the CI does not have to observe the sample behaviors listed in order to rate the student. Finally, there are 2 VASs for rating overall performance. The low end of one VAS is labeled "does not meet expectations," and the high end is labeled "meets expectations." The low end of the other VAS is labeled "novice clinical performance," and the high end is labeled "entry-level performance." Each VAS is marked with an "M" indicating the student's performance at midterm and with an "F" indicating the student's level of performance at the end of the clinical education experience. Space is provided for midterm and final comments from the CI. The student is not required to record any self-assessment, and the tool is designed to be used for only one individual clinical education experience. Use of the CPI does not provide for records from a previous clinical education experience to be carried forward to a new experience.

Physical therapy educators are faced with the choice of which clinical evaluation tool to choose: the CPI, The Blue MACS, or another tool. The evaluation system used by an education program for clinical education experiences may be a major influence on the performance of students.[6] Evaluation instruments that use more global ratings, similar to the CPI, tend, in our opinion, to yield less reliable results[7-11] and to overestimate students' knowledge.[9-11] Visual analog scales with no anchoring descriptors for intermediate points can pose a problem of interpretation for intermediate markings.[12] The mark for "entry level" is clear and would presumably be required at the conclusion of the final clinical education experience. However, if there is more than one clinical education assignment, how are marks somewhere between the anchoring descriptors interpreted for the earlier assignments? How much change in a VAS rating represents a noticeable improvement?

Experience with The Blue MACS, in our opinion, has resulted in benefits for education programs, students, and CIs in Texas and other states. Representatives of the education programs communicate with the center coordinators of clinical education (CCCEs) about which skills and key indicators from The Blue MACS are appropriate for each student to attempt in their centers and about which skills and key indicators may be approved on a clinical education assignment to their facility. Predetermined criteria can then be used to provide direction to the CI about each student's level of preparation and the expectations for student performance for that particular clinical education experience.[1,2] With the CPI, information about each student's previous classroom preparation would have to be communicated in a different format.

A single copy of The Blue MACS is used by the student in each clinical education experience. Therefore, the CI has access to information about the student's previous experiences. Such information assists the CI in planning a clinical education experience that will be most beneficial to the student's level of preparation and complimentary to the student's previous clinical experience.[1] A new CPI is used for each clinical education experience, which may allow the student to start with a "clean slate" if the academic coordinator of clinical education (ACCE) or the director of clinical education (DCE) does not convey information about previous clinical education experiences to the CCCE. If there is concern that the CI's ratings may be influenced by information from previous clinical education experiences, it may be beneficial to use a new, unmarked CPI or other evaluation instrument for each clinical education experience, especially if the student had a previous, unsuccessful experience.

With The Blue MACS, students have a list of predetermined objectives that they must achieve during each clinical education experience, which may assist them in directing their clinical learning.[1,2,13] With the CPI, student objectives for the particular clinical education experience would have to be developed separately by the student, CI, ACCE, or CCCE. In the absence of such objectives, we believe it is likely that the student and CI may miss opportunities to involve the student in new experiences.

The student is required by The Blue MACS to self-evaluate each key indicator, as recommended by guideline 6.3 of the APTA Guidelines and Self-Assessments for Clinical Education.[14] According to Kern and Mickelson,[1] self-evaluation often reveals that students have a very clear idea of their own level of performance. If there is a discrepancy between the student's rating and the CI's rating, this difference should stimulate discussion.[2,4] With the CPI, the student is not required to self-evaluate.

Use of The Blue MACS facilitates formative evaluation by encouraging frequent comparison of student progress with the required competencies.[2,4,6] The CPI provides for evaluation only at midterm and the final evaluative meeting.

Currently, many physical therapist education programs are in the process of making a decision about whether to use the CPI, The Blue MACS, or another tool. For that reason, we believe that it is essential to carefully evaluate the characteristics of each tool. It has been suggested that the success of a clinical assessment instrument will be determined by its acceptance by clinical educators intending to use it[13] and that there is widespread dissatisfaction with clinical education evaluation instruments.[5,9,10,15-19] A major purpose of our study was to investigate the level of acceptance for The Blue MACS by CIs who are familiar with the tool. In addition, we argue that it is critical that the instrument be used consistently so that the education program receives comparable information about the performance of each student. We asked CIs to report whether they consistently follow 10 written instructions from The Blue MACS. Although an investigation of reliability was beyond the scope of our study, we believe that CIs' perceptions of their own consistency in following the written instructions for use provide a basis for reliability. Self-report data may not always accurately reflect actual practice, but self-report measures are in widespread use for initial investigation within a field of study.[20]

Finally, we assessed whether experience as a CI, training in a workshop, or practical experience using The Blue MACS appears to affect attitudes or reported adherence to the instructions for use of The Blue MACS. Various authors have suggested that CIs should be educated carefully and completely in the use of the clinical education evaluation instrument,[1,21,22] but there has been little data on the effects of workshop training or practical experiences on reported attitudes and the manner in which The Blue MACS is used.

Method

Study Design and Subjects

Questionnaires were mailed to all 177 physical therapy clinical education affiliates of Texas Tech University Health Sciences Center (TTUHSC). To ensure responses from CIs with different levels of experience, the cover letter requested that the most and least experienced CIs at each site, or the only CI if the site did not have more than one CI, complete and return the questionnaire. To be eligible to participate in the study, the CI must have used the fifth edition of The Blue MACS. Consent was implied by returning the questionnaire.

Instrumentation 

A 42-item investigator-generated questionnaire was developed to address the study questions. There were 3 parts to the questionnaire. The first 10 questions (part 1) gathered demographic data including experience as a physical therapist, as a CI, and specifically with The Blue MACS. The CIs were asked to estimate the average amount of time they required to complete a student evaluation using The Blue MACS and to report what type of training they had received in using the instrument. The CIs were also asked to rate The Blue MACS in comparison with other clinical evaluation instruments they had used as "much better," "somewhat better," "not better or poorer," "somewhat poorer," or "much poorer."

Part 2 of the questionnaire used a 5-point Likert scale, with responses ranging from "strongly agree" to "strongly disagree," and consisted of 22 questions assessing satisfaction or dissatisfaction with the instrument. It included statements assessing the perceived effectiveness of The Blue MACS as a measure of entry-level performance in various areas, user-friendliness, and helpfulness for planning the clinical experience.

Part 3 used a similar 5-point Likert scale, with responses ranging from "always" to "never," and consisted of 10 questions drawn from the directions in the front of The Blue MACS to assess the degree to which CIs would report that they adhere to the instructions when they use the instrument. Some questions were phrased positively and some questions were phrased negatively in parts 2 and 3 to reduce any tendency to assume a response set that would result in all questions being answered with the same rating without careful consideration of each question.

Scoring

Each question was given a score of 5 to 1, corresponding to "a" ("strongly agree" for part 2 and "always" for part 3) to "e" ("strongly disagree" for part 2 and "never" for part 3), when the question was phrased positively and a score of 1 to 5, corresponding to "a" to "e", when the question was phrased negatively. A total score for each CI was calculated for the opinion scale (part 2) and for the adherence scale (part 3) by adding the scores for each question. Thus, the possible range of scores for part 2 was 22 to 110. For part 3, the possible range of scores was 10 to 50.

Definitions for Opinion

In the opinion section, a score of 88 would result if all responses to positively worded statements in part 2 were "agree" and all responses to negatively worded statements were "disagree." For this reason, we defined scores of 89 to 110 as representing a "very positive opinion." If all questions in the opinion section were answered "neutral," the score would be 66, so scores ranging from 67 to 88 were defined as "positive." In a similar manner, scores of 44 to 66 were defined as "negative" and scores of 22 to 43 were defined as "very negative."

Definitions for Adherence

In part 3, a score of 50 would result if all responses to questions that concurred with the directions in The Blue MACS were marked "always" and all responses to questions that described nonadherence with instructions for use of The Blue MACS were marked "never." We defined scores of 41 to 50 as "very consistent adherence." If all questions in the adherence section were answered "occasionally," the score would be 30. Scores of 31 to 40 were defined as "moderately consistent adherence." In a similar manner, scores of 30 or below were defined as "inconsistent adherence."

Survey Characteristics

Face validity of items on the questionnaire was assessed by 2 academic coordinators of clinical education (ACCEs) and 12 CIs who were experienced in using The Blue MACS but who were not associated with this investigation. Test-retest reliability was assessed by 13 CIs who filled out the questionnaire twice. The 2 survey instruments were distributed 1 week apart.

Test-Retest Reliability

Intraclass correlation coefficients (3,1) of .78 for total scores on part 2 and .83 for total scores on part 3 were computed for test-retest agreement, using the method of Portney and Watkins.[20(p565)]

Procedure

We mailed a packet including an instructional cover letter, a demographic data sheet, 2 copies of the coded questionnaire, 2 coded Scantron sheets, and a self-addressed, stamped envelope to each clinic. The CIs were asked to fill out the demographic data sheet and the Scantron form and to return the completed forms in the provided envelope. To ensure confidentiality, names of participants or facilities were not attached to the questionnaires.

Data Analysis

Descriptive statistics for demographic data were computed as means and percentages. Scores for parts 2 and 3 were computed as described earlier and categorized according to the definitions. Percentages were computed for descriptive statistics.

Scores for part 2 (opinion) were analyzed by a 3-way analysis of variance (ANOVA) to investigate whether experience as a CI, training in a workshop, or practical experience with The Blue MACS would affect attitudes. The independent variables used were 3 items from the demographic data: whether the CI had attended a Texas Consortium for Physical Therapy Clinical Education workshop for training in use of the tool, whether the CI had been graded as a student using The Blue MACS, and the number of students the CI reported having supervised. Many CIs reported supervising a range of students (ie, 15-20), in which case the mean of the 2 numbers was computed and entered into the data analysis as the number of students supervised. The median (between 9 and 10 students) of the range was used to group more and less experienced CIs for data analysis. Many CIs reported supervising "more than" a certain number of students. In those cases, the number recorded was used. A 3-way ANOVA using scores on part 3 (reported adherence) as the dependent variable was performed using the same independent variables in order to investigate whether experience as a CI, training in a workshop, or practical experience with The Blue MACS would affect reported adherence. The hypothesis was that practical experience being graded as a student, experience as a CI, or attending a Texas Consortium for Physical Therapy Clinical Education workshop would result in improved opinions and greater reported adherence, as measured by parts 2 and 3 of the questionnaire.

The ANOVAs were computed on summary scores for parts 2 and 3, not directly on the Likert scale data. Although it could be argued that the summary scores are also ordinal data because the intervals cannot be demonstrated to be equal, it could also be argued that the intervals are equal because each interval represents one Likert scale point. We believe that the spread of scores computed and used in the ANOVAs more closely resemble interval data than ordinal data. An argument for this approach can be found in Kerlinger's Foundations of Behavioral Research.[23]

Results

Demographics (Part 1)

The facility response rate was 57% (101 out of the 177 facilities to which questionnaires were mailed). Thirteen of the 101 facilities responded that they were unable to participate in the study because they had not used the fifth edition of The Blue MACS. One hundred twenty-four questionnaires were returned with part 2 completed by a physical therapist CI, and 117 questionnaires were returned with part 3 completed by a physical therapist CI. It was not possible to determine the response rate for individuals because the number of facilities with only one CI (or no CI) at the time of the survey was not known.

The respondents reported an average of 9.9 years as a physical therapist (SD=8.6 years, range=11 months to 43 years) and an average of 5.9 years as a CI (SD=6.0 years, range=6 weeks to 35 years). The CIs reported being affiliated with an average of 6.2 physical therapy programs other than TTUHSC (SD=5.3, range=0-28) and 1.9 physical therapist assistant programs (SD=2.1, range=0-18). The mean number of students instructed by each CI, as reported by the respondents, was 14.8 (SD=19.1, range=1-144). The respondents reported a large variety of types of facilities in which they served as CIs.

Eighty-four (67.7%) of the 124 CIs had been graded using The Blue MACS when they were students. Seventy-two CIs (58.5%) had attended a formal training session in use of The Blue MACS sponsored by the Texas Consortium for Physical Therapy Clinical Education, and 53 (43.1%) of the 123 CIs who responded to this question had used instruments other than The Blue MACS to grade students on clinical education experiences.

Opinions of the Blue MACS (Part 2)

Part 2 assessed CIs' opinions of The Blue MACS. The majority of the respondents rated each positively worded statement as either "agree" or "strongly agree" and each negatively worded statement as either "disagree" or "strongly disagree." The mean total score for responses to part 2 was 87.1, falling near the upper limit of the range defined as overall positive (67-88). Scores on part 2 ranged from 61 to 109. Three CIs (2.4%) had scores lower than 67 (negative). No response scored in the very negative range (22-43). Sixty-nine respondents (55.6%) had a score of 67 to 88 (positive), and 52 respondents (41.9%) scored over 88 (very positive) (Fig. 2).

[Figure 2 ILLUSTRATION OMITTED]

The 3-way ANOVA resulted in one main effect and no interaction for the 106 questionnaires with complete information. The CIs who had attended a Texas Consortium for Physical Therapy Clinical Education workshop viewed The Blue MACS more positively ([bar]X=89.2, SD=8.81) than those who had not attended a workshop ([bar]X=84.2, SD=9.4) (F=11.4, df=1, P [is less than] .001). The number of students supervised or having been graded as a student using The Blue MACS had no effect on opinion.

Reported Adherence to Instructions for Use of The Blue MACS (Part 3)

Part 3 assessed the CIs' self-reports of adherence to instructions for the use of The Blue MACS. The majority of the respondents rated each statement indicating adherence as either "always" or "frequently" and each statement indicating nonadherence as either "rarely" or "never." The mean total score was 41.9 (SD=4.8) out of a possible range of 11 to 50, falling near the lower limit of the range defined as very consistent adherence (41-50). Participants' responses ranged from 26 to 50. As shown in Figure 3, 3 (2.6%) of 117 respondents achieved scores of 30 or below, which we defined as inconsistent adherence. Forty-one respondents (35.0%) had scores of 31 to 40 for part 3, which we defined as moderately consistent adherence. Seventy-three respondents (62.4%) scored over 40 for part 3, which we defined as very consistent adherence.

[Figure 3 ILLUSTRATION OMITTED]

The ANOVA revealed no interactions and no main effect for reported adherence computed on 110 responses with complete information for all 3 independent variables and for the dependent variable. The CIs who had attended a Texas Consortium for Physical Therapy Clinical Education workshop did not score differently for adherence to instructions for use of The Blue MACS (part 3) from CIs who had not attended a Texas Consortium for Physical Therapy Clinical Education workshop. Scores for adherence did not differ between CIs who had supervised more than 9 students (the median number) and those who had supervised 9 or fewer students. Finally, reported adherence did not differ for those who were or were not graded using The Blue MACS as a student.

Comparison With Other Instruments

Fifty-three (43.1%) of the 124 respondents indicated that they had used clinical assessment instruments other than The Blue MACS. Fifty-one of the 53 CIs who had used other instruments compared The Blue MACS with other instruments. Of those 51 respondents, 25 CIs (49%) responded that they thought The Blue MACS was "much better" than other instruments, and 17 CIs (33.3%) responded that The Blue MACS was "somewhat better" than other instruments. Only 9 CIs (17.6%) responded that they thought The Blue MACS was "no better" or "poorer" than other instruments. None of the CIs indicated that The Blue MACS was "somewhat poorer" or "much poorer" than other assessment instruments that they had used. Data for this study were collected in 1996, before the CPI[5] was available.

Positive Opinion and Adherence

The CIs' reports of adherence to instructions for use of The Blue MACS were related to whether they had a positive opinion of the instrument (r=.40), although the correlation was not a strong one.

Time to Complete Instrument

The CIs' reports of time to complete The Blue MACS ranged from 20 minutes to 3 days. Of the 123 CIs who responded to this question, 121 reported that 5 hours or less is needed to complete the instrument. The 2 responses above 5 hours (2 days and 3 days) were outliers according to the Dixon test[24] ([r.sub.22]=.937, P [is less than] .005; [r.sub.22]=.905, P [is less than] .005) and were discarded from further analyses. The average amount of time reported to complete The Blue MACS with the outliers omitted was 1.59 hours (SD=0.89). There was no correlation between the amount of time reported to complete The Blue MACS with part 2 (opinion) (r =.03) or with part 3 (adherence) (r =.03) of the evaluation. The CIs who had been graded on their own student clinical education experiences using The Blue MACS did not differ in the amount of time required to complete the tool ([bar]X= 1.56 hours, SD=0.94) from those who had not been graded on their own student clinical education experiences using The Blue MACS ([bar]X=1.65 hours, SD=0.87). The CIs who had attended a Texas Consortium for Physical Therapy Clinical Education workshop did not differ in the amount of time needed to complete the evaluation ([bar]X=1.67 hours, SD=0.92) from those who had not attended a Texas Consortium for Physical Therapy Clinical Education workshop ([bar]X= 1.46 hours, SD=0.93).

Discussion

The overall high ratings reveal general satisfaction with the instrument among clinical educators for the Texas Consortium for Physical Therapy Clinical Education. The CIs surveyed have a generally positive opinion of the Blue MACS and believe it to be a useful measure of student clinical performance. In the study, however, CIs were chosen based on their affiliation with an educational institution in Texas that uses The Blue MACS. Generalization, therefore, is limited.

The results of our study also indicate that the majority of CIs report using the instrument according to its written instructions, although actual adherence cannot be determined from our data. These results agree with the findings of studies that showed higher consistency of use with instruments that define performance expectations with competency-based, criterion referenced statements, comparable to those in The Blue MACS.[6,13,21,25-27]

Experienced and inexperienced CIs have similarly positive opinions of The Blue MACS, regardless of whether they had been graded as a student using the tool. Attending a workshop presented by the Texas Consortium for Physical Therapy Clinical Education appears to foster a more positive opinion of the instrument. The importance of general approval should not be underestimated, because a positive opinion of the evaluation instrument may encourage becoming and continuing as a CI. Additionally, a positive attitude toward the instrument, as measured by part 2, was correlated with reports of adherence to instructions, as measured by part 3. We believe this finding indicates that acceptance of a clinical education evaluation instrument may facilitate appropriate implementation of the recommended evaluation process.

The CIs with different levels of experience as CIs, regardless of whether they had been graded as a student using The Blue MACS or whether they had attended a Texas Consortium for Physical Therapy Clinical Education workshop, did not differ in their self-reports of correct use of the instrument. Evidence from our study suggests that the amount of time required to complete the evaluation does not influence the CI's opinion of the instrument, nor does adhering to the written instructions require the CI to spend more time completing the evaluation.

Clinical instructors who have a positive opinion of The Blue MACS report using it correctly more consistently than those who have a negative opinion of the instrument. We do not know whether having a positive opinion encourages the CI to report using the tool more consistently in the correct manner or to actually use the tool more consistently in the correct manner, or whether using the tool correctly causes the CI to have a better opinion of the instrument. Nevertheless, the fact that the positive opinions of the instrument and self-reports of adherence to instructions are correlated suggests to us the importance of promoting a positive opinion of a clinical evaluation tool as well as promoting correct usage.

Limitations of the Study

Our study was limited to Texas Consortium for Physical Therapy Clinical Education affiliates. Most of the respondents were employed in Texas, and approximately 67.7% of the respondents were educated in Texas. Therefore, the results cannot be generalized to CIs employed in other areas of the country or educated in other states. Sixty-eight percent of the CIs had been graded using The Blue MACS as a student, and 58.5% had some training in use of The Blue MACS. Thus, results might differ for CIs with less familiarity with or less preparation for using the tool. Attendance at a Texas Consortium for Physical Therapy Clinical Education workshop was associated with a better opinion of the tool (part 2), but not with adherence to instructions for use of The Blue MACS (part 3). The CIs who used the tool as a student or who were educated in Texas did not have higher scores on part 2 (opinion) or part 3 (adherence) than those who had not used The Blue MACS as a student or who had not attended a Texas school. These 2 measures of familiarity with The Blue MACS (using the tool as a student or attending a Texas school) did not affect opinions or reported adherence, and attending a workshop affected only opinion, not reported adherence. Thus, evidence from our study does not suggest that familiarity with the tool necessarily improves the CI's opinion of the tool or adherence to instructions for use of the tool.

We did not have complete information about the number of potential respondents at each clinical site; therefore, it was impossible for us to determine the response rate for individual CIs. The response rate for facilities was low (57%), which may limit the generalizability of the results. Finally, the test-retest study was limited by the small number of respondents.

Directions for Further Investigation

Now that the CPI is available, clinicians who have used both The Blue MACS and the CPI could be polled as to their preferences. More importantly, evidence regarding the interrater reliability and predictive validity for clinical education evaluation instruments in general and for The Blue MACS and CPI in particular are needed to validate their use for grading students during their clinical education experience. Our study needs to be expanded to include CIs working and residing in other areas of the country, CIs primarily serving students other than those from Texas physical therapist education programs, CIs not educated as students using The Blue MACS, and CIs who have not participated in a Texas Consortium for Physical Therapy Clinical Education workshop. In addition, the validity of self-reports of adherence to instructions for use of The Blue MACS needs to be studied.

Conclusion

A large majority of participating Texas CIs reported that they use The Blue MACS correctly and have an overall positive opinion of the instrument. The findings of our study indicate that formal training results in a more positive opinion of the tool. The majority of the CIs affiliating with the Texas Consortium for Physical Therapy Clinical Education did not find The Blue MACS to be excessively time-consuming to complete and generally preferred it to other tools they had used. Opinions of the tool were better among respondents who had attended a workshop to learn to use the instrument.

Schools currently considering clinical education evaluation instruments should review the evidence available regarding acceptability to clinical educators, whether their use as a clinical education evaluation tool is implemented as intended in the clinical setting, and any evidence regarding reliability and validity. The results suggest that physical therapist education programs should consider offering formal education for use of their clinical education evaluation instrument. Evidence from our study suggests a high level of acceptance of The Blue MACS and reasonably good levels of reported adherence to the written instructions for using The Blue MACS.

References

[1] Kern BP, Mickelson JM. The development and use of an evaluation instrument for clinical education. Phys Ther. 1971;51:540-546.

[2] Nethery J. Physical therapy competencies in clinical education. Phys Ther. 1981;61:1442-1446.

[3] Radtka S, Dragotta N, Needham S, et al. Texas Consortium for Physical Therapy Clinical Education: a model for interinstitutional consortium arrangements. Phys Ther. 1983;63:971-974.

[4] Texas Consortium for Physical Therapy Clinical Education Inc. The Blue MACS: Mastery and Assessment of Clinical Skills. 6th ed. Austin, Tex: Texas Physical Therapy Education and Research Foundation; 1998.

[5] Physical Therapist Clinical Performance Instrument. Alexandria, Va: American Physical Therapy Association; 1998.

[6] Petrusa ER, Blackwell TA, Rogers LP, et al. An objective measure of clinical performance. Am J Med. 1987;83:34-42.

[7] Hull AL, Hodder S, Berger B, et al. Validity of three clinical performance assessments of internal medicine clerks. Acad Med. 1995; 70:517-522.

[8] O'Donohue WJ Jr, Wergin JF. Evaluation of medical students during clerkship in internal medicine. J Med Educ. 1978;53:55-58.

[9] Ramsey PG, Shannon NF, Fleming L, et al. Use of objective examinations in medicine clerkships: ten-year experience. Am J Med. 1986; 81:669-674.

[10] Wood V. Evaluation of the student nurse clinical performance: a continuing problem. Int Nurs Rev. 1982;29:11-18.

[11] Marienfeld RD, Reid JC. Subjective vs objective evaluation of clinical clerks. N Engl J Med. 1980;302:1036-1037.

[12] Borg G. Borg's Perceived Exertion and Pain Scales. Champaign, Ill: Human Kinetics Inc; 1998.

[13] Newble DI. Eight years' experience with a structured clinical examination. Med Educ. 1988;22:200-204.

[14] Guidelines and Self-Assessments for Clinical Education. Alexandria, Va: American Physical Therapy Association; 1999:46.

[15] Tonesk X. Clinical judgment of faculties in the evaluation of clerks. J Med Educ. 1983;58:213-214.

[16] Curry L, Wergin J. Educating Professionals: Responding to New Expectations for Competence and Accountability. San Francisco, Calif: Jossey-Bass Inc Publishers; 1993.

[17] Friedman M, Mennin SP. Rethinking critical issues in performance assessment. Acad Med. 1991;66:390-395.

[18] Kreiter CD, Ferguson K, Lee WC, et al. A generalizability study of a new standardized rating form used to evaluate students' clinical clerkship performances. Acad Med. 1998;73:1294-1298.

[19] Bondy KN. Criterion-referenced definitions for rating scales in clinical evaluation. J Nurs Educ. 1983;22:376-382.

[20] Portney LG, Watkins MP. Foundations of Clinical Research: Applications to Practice. Upper Saddle River, NJ: Prentice Hall Health; 2000:287.

[21] Mays MJ. Reliability of a method of evaluating the clinical performance of a physical therapy student. Phys Ther. 1973;53:1289-1306.

[22] Fitzpatrick JM, While AE, Roberts JD. Operationalisation of an observation instrument to explore nurse performance. Int J Nurs Stud. 1996;33:349-360.

[23] Kerlinger FN. Foundations of Behavioral Research. 2nd ed. New York, NY: Holt, Rinehart and Winston Inc; 1973:286-288.

[24] Dixon WJ. Processing data for outliers. Biometrics. 1953;9:74-89.

[25] Harden RM, Gleeson FA. Assessment of clinical competence using an objective structured clinical examination (OSCE). Med Educ. 1979; 13:41-54.

[26] Stillman PL, Ruggill JS, Ratala PJ, Sabers DL. Patient instructors as teachers and evaluators. J Med Educ. 1980;55:186-193.

[27] Andrew BJ. The use of behavioral checklists to assess physical examination skills. J Med Educ. 1977;52:589-591.

J Hrachovy, PT, MSPT, and N Clopton, PT, PhD, are faculty members at Texas Tech University Health Sciences Center, Lubbock, Tex. K Baggett, PT, MPT, T Garber, PT, MPT, J Cantwell, PT, MPT, and J Schreiber, PT, MPT, are recent graduates of the Physical Therapy Program, Texas Tech University Health Sciences Center. Address all correspondence to Dr Clopton at: Physical Therapy Program, Texas Tech University Health Sciences Center, 3601 4th St, Lubbock, TX 79430 (USA) ([e-mail]).

All authors provided concept/research design and writing. Ms Hrachovy, Ms Baggett, Ms Garber, Ms Cantwell, Ms Schreiber, and Douglas Hubbard provided data collection. Ms Hrachovy, Dr Clopton, Ms Garber, Ms Cantwell, Ms Schreiber, Mr Hubbard, Steve Sawyer, and James Clopton provided data analysis. Ms Hrachovy, Ms Baggett, Ms Garber, and Ms Schreiber provided project management and fund procurement; Ms Cantwell also assisted with fund procurement. Ms Hrachovy provided subjects, and Ms Baggett, Ms Garber, and Ms Schreiber provided clerical support. Dr Clopton and Mr Clopton provided consultation (including review of manuscript before submission).

This study was approved by the Institutional Review Board for the Protection of Human Subjects at Texas Tech University Health Sciences Center. The results of this study were presented, in part, at the Combined Sections Meeting of the American Physical Therapy Association, February 12, 1998, Boston, Mass.

This study was supported by a research grant from the Texas Physical Therapy Education and Research Foundation. There is a potential conflict of interest in that the Texas Physical Therapy Education and Research Foundation is funded, in part, by sales of The Blue MACS. In addition, Ms Hrachovy is currently chair of the Texas Consortium for Physical Therapy Clinical Education, which developed The Blue MACS.

This article was submitted February 9, 1999, and was accepted March 21, 2000.