Comprehensive practicum evaluation across a nursing program.
THE EVALUATION OF CLINICAL PERFORMANCE is a challenge for all health disciplines, as well as for nursing education. In contrast to didactic course evaluation, which is traditionally grounded in objective testing strategies to determine cognitive achievements, clinical courses require evaluation of cognitive, affective, and psychomotor achievements. The evaluation of progressively complex clinical skills presents an additional challenge. * This article reports on a laboratory and clinical evaluation plan that follows the progressively difficult skill sets required in successive semesters of a baccalaureate nursing program. The article addresses the evolution of the evaluation plan, the resources needed to implement the plan, progress to date, and future directions. The term practicum used in this article refers to either a laboratory or clinical-based setting.
Literature Review Despite widespread acknowledgment that clinical evaluation is, to a large extent, subjective in nature, there has been a strong movement toward criterion-referenced, away from norm-referenced, evaluation. Use of criterion-referenced evaluation tools is regarded as an opportunity to remove some of the subjectivity from clinical evaluation (1,2).
Growing out of the recent emphasis on criterion-based evaluation has been the development of "standardized" evaluation tools, such as the Objective Structured Clinical Examination (OSCE) (3) and the modified OSCE (4). The OSCE was developed for medical students, but has been adapted for other health fields, including nursing, and is widely used in England (4-6). The behaviorally focused OSCE is frequently used as a summative evaluation tool, although some have proposed using it for formative evaluation (7).
Other tools reported in the literature include the Clinical Performance Manual (CPM) (8), the Clinical Evaluation Tool (CET) (9), and the Clinical Competence Criteria Valuing Scale (CCCVS) (10). Acceptable reliability and validity scores have been demonstrated by these tools. The CPM is reported to have an 85 percent agreement rating by students with regard to the tool's ability to evaluate skills, and a 100 percent agreement rating by faculty to establish content validity for tested skills in nursing students at a variety of levels (8). The CET demonstrated a Cronbach's alpha ranging from .83 to .97 and correlation coefficients ranging from .29 to .69 on variables analyzed in senior-level students in a four-year nursing program (9). The CCCVS, when compared to the Clinical Competence Rating Scale (CCRS) (11), established an alpha level of .9575 in faculty and preceptors with students in the third and fourth years of a four-year nursing program (10).
Despite acceptable reliability and validity scores, some tools have been criticized for their reductionist approach. Specifically, these tools have measures that are so narrowly written that they may not allow for a more comprehensive, context-sensitive approach to evaluation (1,12). The use of computer simulations (13), vignettes (14), and patients in the evaluation process has also been reported (15).
More recent writings emphasize the dual faculty role of teacher and evaluator. Because the two roles often cannot be separated, attention has been given to the teacher as a tool for inquiry, similar to the notion of the qualitative researcher as the data collection tool, with regard to consistency across evaluators (16). In concert with this viewpoint, Neary (17) contends that continuous assessment of the student, rather than a single observation session, is the preferred method for clinical evaluation to ensure accurate evaluation data.
While much has been written about evaluation within a single course, a growing issue is that of determining whether students are meeting progressively increased expectations in subsequent courses. Related to this issue is the importance of the development of an evaluation program that requires students to continue to validate previously achieved skills while demonstrating newly learned skills. No literature was uncovered that addresses these two critical points. This article addresses the gap in literature related to practicum evaluation by specifically targeting increased competency expectations across a nursing program.
The Impetus for a Comprehensive Evaluation Program As with many other four-year baccalaureate schools of nursing, the Indiana University School of Nursing at Bloomington recently adopted a new competency-based curriculum reflective of the National League for Nursing Accrediting Commission and American Association of Colleges of Nursing statements on graduate competency (18,19). A competency-based program is grounded in the development of core end-of-program outcomes. Specific, measurable "year-level" competencies for each program year (sophomore, junior, and senior) are selected to demonstrate the achievement of program outcomes.
Because of the change in the curriculum, faculty recognized the need to have objective measures of every clinical student. An additional concern was the recent "borderline" performance of students in clinical experiences. Faculty desired an improved process for determining the performance necessary to progress to the next semester. In addition, a movement toward assigning letter grades, rather than a satisfactory/fail grade, was being considered.
Of particular interest was a change in the physical assessment and psychomotor skills courses from laboratory-based only to both laboratory and real-patient experiences. This change led to a need to collect data on a sufficient number of correct skill performances in both the laboratory and patient care settings.
The faculty were familiar, through conferences, with the pioneering work of Alverno College in Wisconsin in developing comprehensive evaluation tools based on the demonstration of desired behaviors. Several faculty workshops were arranged with the dean of the Alverno College Department of Nursing. At subsequent workshops, following the initiation of the practicum evaluation, faculty again worked intensively with the Alverno dean to evaluate the experience and design the next level for the experience.
It is important to note that several faculty were allotted teaching credit for the development and implementation of the practicum. This was considered crucial since nursing faculty workloads are traditionally high due to clinical components. Had no time resources been allowed, it is likely that faculty would not have been sufficiently invested in the evaluation and the effort would not have been successfully implemented.
Description of the Evaluation Program The initial evaluation effort was confined to one clinical course in one semester and was entirely laboratory based. Its success motivated faculty to move toward developing similar experiences across four semesters in the sophomore and junior years. The current program spans four semesters, combines skills from prerequisite and concurrent courses, and encompasses both laboratory and clinical-based practica.
One faculty member received teaching credit for working with colleagues to ensure consistency across courses. The courses targeted were Health Assessment, Science and Technology of Nursing, Alterations in Health I, and Alterations in Health II. The evaluation program was tailored to the competencies and developed by faculty at the inception of the new curriculum. Each evaluation tool reflects the appropriate year-level competency expected for students.
Evaluation criteria consistent across all courses were: critical thinking (including priority setting), health assessment, psychomotor skills, and communication (including patient teaching and documentation). Table 1 summarizes the progression of the practicum components for each course and the performance activities associated with each practicum.
First Semester, Sophomore Year This semester marks the beginning of the nursing program. Students take Health Assessment, which contains both a didactic and practicum component. Although the practicum component of this course is primarily laboratory based, students also have experiences with direct patient care after mastering a certain number of basic skills in the laboratory. Each week, students demonstrate individual skills for faculty members.
The practicum evaluation at the end of the semester is laboratory based and uses junior-level student actors as "patients." Course faculty are responsible for the evaluation of student performances. Students who are being evaluated receive a written case study about a patient that includes the presenting complaint. They are then expected to perform a head-to-toe assessment, as well as a more focused assessment of the presenting complaint. For example, if a patient complains of swelling in her or his feet, a more detailed assessment of the cardiovascular system is warranted.
Second Semester, Sophomore Year The Science and Technology of Nursing is the practicum-based course in this semester. This course has a didactic component as well as a laboratory experience that focuses on acquisition of psychomotor skills such as medication administration, nasogastric (NG) tube insertion, chest tube maintenance, Foley catheter insertion, and intravenous (IV) catheter insertion. As in the Health Assessment course, students learn skills in the laboratory setting and are evaluated each week. They then perform these skills on patients in the clinical setting.
The practicum evaluation is laboratory based and case study based, and skills are performed on mannequins. The evaluation is focused on skills attained in the course and appropriate for the program level year (sophomore). All faculty in the School of Nursing, rather than course faculty only, serve as evaluators of student performance.
During the evaluation, students enter the laboratory and are given a brief overview of the patient's presenting complaint, an abbreviated history and physical, a kardex with treatments listed, and medication sheets. The student draws from two stacks of cards, one that is medication focused and the other that specifies a psychomotor skill to be performed, such as NG tube insertion, starting an IV, inserting a Foley catheter, or changing a dressing. Students must decide which skill they are going to perform first. For example, if a patient complains of nausea but also needs a routine dressing change, the student would be expected to initiate nausea interventions, including medications, and then evaluate the effect of the interventions before initiating the dressing change.
First Semester, Junior Year In this semester, the practicum evaluation is a part of the Alterations in Health I course, the first full medical-surgical course in the nursing curriculum. The evaluation takes place approximately two thirds of the way into the semester to allow for remediation. Students are required to design, implement, and evaluate nursing care to patients on medical-surgical units in the hospital. To execute this care, students must use all skills from earlier semesters and make connections among preexisting and current patient conditions.
As with the second-semester, sophomore-year evaluation, this evaluation is laboratory based and case study based, and all faculty on staff serve as evaluators of student performance. In addition, the case incorporates family health content (pediatrics, obstetrics) or mental health, depending upon which clinical course the student is taking concurrently with the alterations course. Sophomore-level students serve as "patients."
The setup for this evaluation is most like that of the initial practicum evaluation and reflects the expected increase in critical thinking ability from the sophomore to the junior year. Students receive a case upon entering the laboratory. They must prepare a prioritized plan of care based on the patient information provided and then draw cards from two stacks. One card will specify a medication skill and the other either a skills procedure or patient teaching. Sophomore-level students serve as patient actors.
There have been semesters in which second-semester, junior-year students have worked on the evaluation team with faculty to evaluate the performance of a first-semester, junior-year student. Because of changes in admission sequencing, this practice is no longer in place. However, if faculty extend the evaluation to the senior year, seniors may be asked to assist in the evaluation of junior students.
In addition to the practicum evaluation, students must complete a pharmacology examination, an electrocardiogram evaluation, a laboratory results examination, and a skills inventory. The skills inventory consists of psychomotor skills that must be performed on patients throughout the course of the semester. If students are not presented with the opportunity to perform these skills during their regular clinical rotation, they are required to validate them to course faculty at the end of the semester. Along with weekly clinical performance, these five components of the practicum provide faculty with a comprehensive evaluation of the student.
Second Semester, Junior Year The practicum evaluation in this semester is tied to the Alterations in Health II course. Students are required to design, implement, and evaluate care for more complex patients. Often these patients present with multi-system issues that require students to intensify their critical thinking skills.
The practicum evaluation is significantly different from those of previous semesters. It takes place midsemester rather than at the end of the semester; it is clinically based with real patients; and students are evaluated by registered nurse preceptors. RN evaluators are selected by faculty who have directly observed their excellence in clinical practice and their teaching skills with students. The majority are graduates of the Indiana University nursing program.
Students are told in advance when they will be evaluated. On the evaluation day, the student provides the RN evaluator with a report on a specific patient and states what will be assessed and why. The RN accompanies the student and evaluates the student's performance across the following areas: identification and prioritization of assessments, execution of assessment techniques, and assessment findings.
The evaluation criteria reflect the expectation that students are able to critically evaluate a complex patient situation to determine what information is needed from the assessment and are able to conduct an accurate assessment on more complex patients. While in previous semesters, students were expected to execute basic assessment skills, they needed assistance with determining what assessments were needed before feeling comfortable. The expectation of competence in more advanced thinking related to assessment skills is appropriate for this semester.
The timing of the evaluation also reflects the higher expectations for second-semester, junior-year students. Sophomore practicum evaluations took place the week before the final examination week, providing the opportunity for remediation during the week of finals. Having the evaluation at midsemester allows students sufficient time to make up deficiencies before the end of the semester.
Although case standardization is not possible with this evaluation strategy, and students have expressed some concerns, students are encouraged to select challenging patients. Faculty provide assurances that the level of expectation is the same despite the complexity of the patient and that RN evaluators are briefed on the expectations of student performance for different levels of patient acuity. So far, this approach has been successful, and evaluation on a complex patient allows stellar students to demonstrate and document their outstanding abilities.
As with first-semester, junior-level evaluation, the second-semester, junior-level evaluation involves more than a single activity. It also includes a pharmacology examination, an electrocardiogram examination, a laboratory results examination, a comprehensive management plan, an executed patient teaching plan, and a skills inventory. The teaching is performed by students on a patient and is evaluated by course faculty. Teaching is expected to be based on teaching-learning principles and an identified patient need. The skills inventory is the same as for the first semester of the junior year and provides for another skill validation.
Challenges and Recommendations Table 2 provides a thematic overview of the evaluative comments made by students across semesters. Students consistently identified stress and concern with evaluation consistency as a drawback.
Among faculty, the consensus is that the practicum evaluations provide a solid piece of evidence by which to gauge student progress within a semester and across the sophomore and junior years. Faculty have continued refinement of the practicum evaluation program because they believe it accurately assesses critical thinking skills as well as selected psychomotor skills.
The most significant challenge with implementing the practicum evaluations has been faculty time, especially with the initial development. Because the amount of time required did not allow for the participation of the entire faculty, practicum evaluations were developed by one faculty member who worked with colleagues. After the practicum evaluations were developed, they were "decentralized" to course faculty, who accepted primary responsibility for the upkeep and updating of the practicum evaluation experiences. Despite this limitation, discussions about the evaluations have provided faculty and students with a clearer vision of expectations and criteria for student performance.
With time, coordination, and consistency major factors in development, it is difficult to have every faculty member participate in every step of the planning and implementation process. However, if only one or two faculty members are responsible for the plan, the possibility exists that other faculty members will fail to "buy in" to the program. Therefore, it is recommended that any attempt to develop this type of plan be first discussed openly in faculty meetings, with full faculty involvement at each step of the development phases, and that an evaluation plan to determine the effectiveness of the program be constructed. It is important to clearly outline the responsibilities of faculty, especially with any practicum that takes place near the end of the semester.
To address the problem of evaluator consistency across all practicum evaluations, course faculty now discuss more fully what students should be told about the evaluation criteria. Similarly, there have been problems with some faculty using the evaluation sessions as an opportunity to teach during the session. Once this issue was identified, faculty agreed to limit the sessions strictly to evaluation and to provide feedback to the student after the session is completed.
Questions about evaluator consistency when RN evaluators are used continue to arise. However, faculty address this problem by consistently stating the level of expectation required of the student. For course faculty, attempts are made to ensure continuity of evaluators across clinical groups and from semester to semester.
Determining how practicum evaluation grades fit within course grades has led to a certain of amount of anxiety among faculty as well as among students. Initially, students could fail the course solely upon failure of the evaluation. In addition, the semester-five practicum originally involved more than one course, and there was a question about where the grade for the practicum should be assigned.
Procedures have changed. Now, the practicum evaluation is grounded in only one course, counts as one clinical day, and allows for remediation. With students who are considered borderline, the practicum could "make or break" them, but with students who are consistently strong, one unsatisfactory performance will not result in a failure for the course.
The last and most significant challenge to students is their anxiety. Although student anxiety will inevitably persist over the experience, faculty have taken steps to minimize the problem. Introducing the practicum evaluation in the first semester of the sophomore year provides experience for subsequent evaluations. In addition, sophomores participate in the junior-level practicum so they are exposed to the process. Finally, faculty provide students with as much detail about the experience as possible, conduct "run throughs," and distribute the evaluation criteria/grading sheets well in advance. These last steps are crucial since students sometimes state that the lack of a real patient situation prevents them from performing as they normally would in the clinical setting.
Future Development Updating the practicum evaluation based on student and faculty feedback is a constant process. Plans are currently under way to extend the evaluations into senior-level courses. In addition, faculty continue to address challenges such as student stress and perceptions of evaluator consistency, and interventions are evaluated for their effectiveness.
While the comprehensive practicum evaluation has required significant time commitment from faculty, the goal of obtaining a data-based, relevant evaluation of each clinical student has been realized. Full faculty involvement has provided for a more coherent progression and understanding of student expectations across each semester. These evaluation tools are relevant as they have been developed within the context of the competencies for the course and the curriculum and according to faculty values.
Table 1. Summary of Practicum Evaluation Sequence and Components COURSE / SEMESTER SETTING COMPONENTS EVALUATORS Health Assessment Laboratory-based * Case-based Course (B245) with student * Head-to-toe faculty First Semester, actors assessment with Sophomore Year inquiry into specific complaint Science and Laboratory-based * Case-based Total faculty Technology of with mannequins * Medication Nursing (B249) and psychomotor Second Semester, skills Sophomore Year Alterations in Laboratory-based * Case-based Total faculty Health I (H354) with student * Written and skill First Semester, actors and component similar Junior Year mannequins to B249 * Pharmacology exam * Electrocardiogram exam * Laboratory results exam * Skills inventory Alterations in Hospital-based * Patient-based RN preceptors Health II (H362) * Evaluation of Second Semester, skills demonstrated Junior Year with actual patient * Complete management plan * Pharmacology exam * Electrocardiogram exam * Laboratory results exam * Skills inventory * Patient teaching Table 2. Summary of Student Comments by Semester SEMESTER / YEAR LEVEL BENEFITS OF PRACTICUM DRAWBACKS OF PRACTICUM First Semester/ * Provided assessment * Stress of the Sophomore Year of all skills experience learned in course * Concerns about * Comprehensive in consistency of terms of simulating evaluation across true experience faculty Second Semester/ * Provided a scenario * Stress of the Sophomore Year under which to experience demonstrate a skill * Only demonstrated 2 * Everyone knew they skills out of many would have to do a learned in the medication skill semester * Concerns about consistency of evaluation across faculty First Semester/ * Case scenario * Stress of the Junior Year effectively experience evaluated critical * Only demonstrated thinking skills 1-2 skills out of * Forced preparation many possible skills of knowing how to * Expressed desire to do all skills have evaluation on real patients at this level * Perception of differences in difficulty across cases * Concerns about consistency of evaluation across faculty Second Semester/ * Evaluated real-time * Stress of the Junior Year skills normally experience performed on * Concerns about patients consistency of * Real setting used evaluation across * Appreciated RN evaluators detailed feedback from RN evaluators * Control over selection of patient for evaluation process
Keywords Baccalaureate Nursing Education--Clinical Evaluation--Competency-Based Curriculum--Criterion-Based Evaluation
(1.) Chambers, M. (1998). Some issues in the assessment of clinical practice: A review of the literature. Journal of Clinical Nursing, 7, 201-208.
(2.) Milligan, F. (1996). The use of criteria-based grading profiles in formative and summative assessment. Nurse Education Today, 16, 413-418.
(3.) Harden, R. M., Stevenson, M., Downie, W. W., & Wilson, G. M. (1975).Assessment of clinical competence using objective structured examination. British Medical Journal, 1(5955), 447-451.
(4.) O'Neill, A., & McCall, J. M. (1996). Objectively assessing nursing practices: A curricular development. Nurse Education Today, 16, 121-126.
(5.) Ladyshewsky, R., Baker, R., Jones, M., & Nelson, L. (2000). Evaluating clinical performance in physical therapy with simulated patients. Journal of Physical Therapy Education, 14(1), 31-37.
(6.) Tuazon, J.A. (1999). Objective structured clinical evaluation for nursing foundation courses. Philippine Journal of Nursing, 69(3-4), 9-15.
(7.) Khattab, A. D., & Rawlings, B. (2001). Assessing nurse practitioner students using a modified objective structured clinical examination (OSCE). Nurse Education Today, 21, 541-550.
(8.) Woolley, G. R., Bryan, M. S., & Davis, J. W. (1998). A comprehensive approach to clinical evaluation. Journal of Nursing Education, 37, 361-366.
(9.) Krichbaum, K., Rowan, M., Duckett, L., Ryden, M. B., & Savik, K. (1994). The clinical evaluation tool: A measure of the quality of clinical performance of baccalaureate nursing students. Journal of Nursing Education, 33, 395-404.
(10.) Ferguson, L. M., & Calder, B. L. (1993). A comparison of preceptor and educator valuing of nursing student clinical performance criteria. Journal of Nursing Education, 32(1), 30-36.
(11.) Scheetz, L.J. (1989). Baccalaureate nursing student preceptorship programs and the development of clinical competence. Journal of Nursing Education, 28(1), 29-35.
(12.) Girot, E. A. (1993). Assessment of competence in clinical practice: A review of the literature. Nurse Education Today, 13, 83-90.
(13.) Grossman, C. C., & Hudson, D. B. (2001). Rating students' technology generated clinical decision making scores. Nurse Educator, 26(1), 5, 12.
(14.) Van Eerden, K. (2001). Using critical thinking vignettes to evaluate student learning. Nursing and Health Care Perspectives, 22, 231-234.
(15.) Lofmark, A., Hannersjo, S., & Wikblad, K. (1999). A summative evaluation of clinical competence: Students' and nurses' perceptions of inpatients' individual physical and emotional needs. Journal of Advanced Nursing, 29, 942-949.
(16.) Mahara, M. S. (1998). A perspective on clinical evaluation in nursing education. Journal of Advanced Nursing, 28, 1339-1346.
(17.) Neary, M. (2000). Responsive assessment of clinical competence: Part 2. Nursing Standard, 15(10), 35-40.
(18.) National League for Nursing Accrediting Commission. (2003). Interpretive guidelines by program type. [On-line]. Available: http://www.nlnac.org.
(19.) American Association of Colleges of Nursing. (1998). The essentials of baccalaureate education for professional nursing practice. Washington, DC: Author.
Deanna L. Reising, PhD, APRN, BC, is an assistant professor at Indiana University School of Nursing, Bloomington. Lynn E. Devich, MSN, MBA, RN, CS, a clinical assistant professor at Indiana University School of Nursing at the time of this writing, is a critical care resource nurse at St. Francis Hospital, Beech Grove, Indiana. The authors acknowledge the support of Dr. Jean Barrels, former dean of the Department of Nursing, Alverno College, Milwaukee, Wisconsin. For more information, contact Dr. Reising at email@example.com.
|Printer friendly Cite/link Email Feedback|
|Author:||Reising, Deanna L.; Devich, Lynn E.|
|Publication:||Nursing Education Perspectives|
|Date:||May 1, 2004|
|Next Article:||Using clinical journaling to capture critical thinking across the curriculum.|