Peer assessment and role play: a winning alliance.

Abstract

Combining peer assessment and role play into one activity can produce an effective assessment method. Incorporating peer assessment into the project results in reduced faculty grading time and increased student learning. This paper describes an assignment that combined the two, the results, and recommendations that faculty can use to readily adapt the technique to their own content areas.

Introduction

Good faculty members are constantly seeking ways to both involve students in classes and increase learning. Long lectures can be boring for students, and studies consistently show that discussion-oriented teaching methods result in increased student engagement and learning. Role play, or simulation, is an established teaching methodology that has been used in education for many years (Bernstein, Scheerhorn, & Ritter, 2002; Taylor & Walford, 1978; Van Ments, 1983). It is viewed as a particularly powerful strategy because it can create learning experiences that are varied and realistic. In role play situations, students are active participants rather than passive observers. Participants must make decisions, solve problems, and react to the results of their decisions. Collier (1998, 2000) has written about the emotional experience for the student and the increased learning as a result of this level of participation.

With the recent increased emphasis on assessment in higher education, a logical extension of using role play or simulation as a teaching method is to use it as an evaluation of student learning. However, a drawback to using it as an assessment tool is the extensive amount of time that is required for the faculty member to review and appraise the performance of each student (Bernstein, Scheerhorn, & Ritter, 2002). Peer assessment as a method to evaluate student performance has increased substantially over recent years as a way to reduce faculty time, and some studies have reported that it can increase the understanding of students conducting the assessment as well (Bangert, 2003). Peer assessment typically involves students exchanging assignments and providing feedback to their peers through the use of rating scales or checklists (Airisian, 1996; Topping & Ehly, 1998). This technique has been used to evaluate student performance in a wide variety of disciplines such as psychology (Haaga, 1993), mathematics (Bangert, 2003; Earl, 1986), biology (Falchikov, 1986), geography (Mowl & Pain, 1995), and computer science (Marcoulides & Simkin, 1991). The project described in this paper combined role play and peer assessment into one assignment used in an education course.

Peer Assessment to Evaluate Role Play

The purpose of the project/assignment described in this paper was to explore the use of peer assessment to evaluate role play in a graduate level higher education administration course. By using peer assessment, I hoped to increase the feasibility and benefit of using role play as an assessment instrument in the classroom. This paper describes the assignment, feedback from students about the value of the project, and lessons learned, and then provides recommendations that faculty can use to implement this technique in their own classes. Feedback from the students was obtained through a written questionnaire completed immediately at the end of the activity.

Description of the Project/Assignment

As a faculty member in an Adult and Higher Education Graduate Program in a 4-year institution located in a rural environment, I decided that our program would provide a good arena to explore combining role play and peer assessment. Students are typically nontraditional age with job experience, and some travel distances of over 100 miles to take courses in the program. Most courses are taught in 3-hour evening time periods or in a weekend format. The assignment was used in "Resource and Program Management," a required course for students in the program, and a variety of topics are covered throughout the semester. Managing human resources is an area of emphasis, and students are presented with principles of listening and communication skills. As an active learning activity, students are given an exercise designed for practicing communication skills as a manager or supervisor in a role play situation. In the past, students were divided into groups of three and asked to practice a scenario involving an employee with problems on the job. One student acted as the manager, another as the employee experiencing difficulty on the job, and the third student was an observer of the role play. Sessions were video taped and reviewed by each group, and then the experience was discussed as a class. No grade was given to the students.

While students reported learning from the practice sessions, I sought to use the simulations as an assessment of student learning of the listening and communication skills presented in class. In an attempt to convert the activity to an assessment technique, in subsequent semesters I tried grading the role play of each student. However, reviewing and evaluating each tape for a grade was found to be too time intensive to be feasible. A review of literature describing the successful use of peer evaluation to assess student learning motivated me to combine the two into one assessment activity. The role play assignment was first used during the spring semester of 2004. Revisions were made to the assignment based upon student reports and my own evaluation of it, and the assignment was given again during the spring semester of 2005.

To begin the project, students were divided into groups of three and given a five-page handout containing all the material needed to complete the assignment. The cover sheet described the activity in detail and delineated how the project was to be graded. The remaining pages included the observer assessment sheet, the self-assessment sheet, the role play scenarios, and, finally, the student evaluation of the project.

The directions explained that one person in each group was to be the manager, another the employee, and the third the observer and time keeper. The person acting as the manager was to practice the listening and communication skills discussed in class in response to the scenarios acted out by the employee. The observer was to take notes during the role play, lead a critique of the session during a replay of the taped session, and complete the observer assessment sheet. The role of each student rotated so that each individual had the opportunity to be a manager, employee, and observer. Students were instructed that each role play was to last a minimum of 5 minutes and a maximum of 8 minutes.

Fifty percent of students' grades was derived from their performance as managers, and 50% was based on their observer roles. The manager portion of the grade was determined by both the actual acting out of this role and the self-assessment. In other words, if a student was dissatisfied with her performance in the role play, she could write up what it would take to improve her performance, and this assessment would be factored into her "manager" grade. The observer role was graded in order to motivate students to give serious and quality feedback. It was also hoped that students would learn how to improve their own behavior by critiquing others.

The observer form was a one-page sheet containing a column that listed each desired behavior (e.g., creating a positive atmosphere, paraphrasing, using open-ended questions, probing statements to encourage additional information, etc.). To the left of each desired behavior were four columns that described various levels of performance exhibited for each behavior. Rather than providing a likert numerical rating, each column provided a short description of the behavior in a rubric style, such as "creates positive atmosphere by beginning with friendly small talk," "begins session in a friendly manner with the business at hand," "begins session with business at hand in a condescending manner," and so forth. During the session and after the group critique, the observer circled the words that best described the behavior of the manager. At the bottom of the page was a space where observers were asked to provide examples of statements made by the manager during the role play. Also, the observer was asked to describe the primary "plus" of the interview as well as a "wish." This feedback structure was designed to keep critiques constructive and to create a positive experience for students.

The manager self-assessment sheet asked the manager to describe "what I did that was a plus" and "my wishes for another time." Students were required to list at least two pluses and one wish so that they were forced to identify positive aspects of their behavior.

Three distinct scenarios were provided for each group, and from these, students were to choose one to use for each role play; alternatively, they were allowed to create their own scenario of a similar nature. The scenarios that were provided were based upon real-life experiences of the author in previous administrative positions.

The evaluation form was a survey that asked for student feedback about the project itself. Students were asked to rate eight statements using a 5-category likert scale of "Strongly agree," "Agree," "Neutral," "Disagree," and "Strongly disagree." The eight statements included the following assertions: "This activity helped me to practice the listening skills discussed in class," "this activity helped me to improve my listening skills by practicing them," "this activity should be used in future classes," and "the observer sheet worked well in assessing how I did." The student evaluation sheet was anonymous and was handed in separately from the other sheets that were to be graded.

All sheets and tapes were collected at the end of class. Each tape was then reviewed in conjunction with both the self-assessment sheet and the observer sheet for each student.

Results

Combining peer evaluation and role play resulted in a positive assessment technique that both reduced the amount of faculty time required to grade students and was rated by students as a beneficial learning experience. I spent approximately 10-15 minutes grading each role play. In previous classes with no peer assessments, time spent grading each role play averaged 25 minutes and in some cases even longer. I found that the student observers provided excellent feedback to the student managers and, in many cases, gave much more in-depth information than I had in the past in my grading. The observers focused on the positive aspects of the managers' behaviors; however, they were not reluctant to offer constructive criticism and ideas for improvement.

The students' self-assessment of their own role play was thorough and well constructed. They provided specific examples of their "pluses" as well as what they "wished" they had said or done. However, they usually listed more "wishes" than "pluses," so it was important to incorporate the group critiques of the session to provide the students with a more complete listing of their positive behaviors.

Student evaluations of the assignment were positive. Likert scores were all in the "agree" or "strongly agree" ratings. All students either "agreed" or "strongly agreed" that the activity should be used in future classes. Specific comments from the students included the following: "I always like to put what I'm learning into practice," "this was a good way to assess where my current skills are and what areas I need to improve," "everyone should take this class and do this activity," and "the critique sessions generated a lot of helpful discussions and gave me ideas that I had not thought about."

Following are some of the student "wishes" that were expressed: "I wish we could have had the opportunity to do the taping again to see if we could incorporate the group's feedback into another meeting," "I think it would have been helpful to tape the critique, as this provided much more feedback on the pluses and wishes section, and then I could have reviewed it later." Also, feedback from the first semester resulted in the observer sheet being reduced from two pages in length to one.

Lessons Learned and Recommendations

Faculty members in other disciplines can readily adapt this assessment technique to their own content areas. Important considerations and recommendations include the following:

1. Small class size is probably most feasible for this assignment for several reasons. First, since each group requires its own equipment, the number of available recording and replay devices should be determined before incorporating this tool into a class. Second, the noise level in the classroom during the completion of the assignment is considerable. During the first semester that I used the technique, some students reported that the other groups' discussions were distracting to them. I found it helpful to use additional classrooms to reduce the number of groups in each room. However, making this an outside-of-class assignment could avoid both of these dilemmas. Third, since faculty still need to review each tape in order to maintain some quality control over the peer assessment, larger classes may be too time consuming. I used the assignment for classes with fewer than 20 students, thus minimizing the amount of time spent grading.

2. Time spent setting up the equipment and instructing students must be taken into consideration. I placed the equipment in the room and checked its operation before class began and needed some assistance from our media department in doing this. Students were fairly knowledgeable in the manipulation of traditional video taping and replay equipment; however, as colleges transition to digital recording devices, students may need additional instruction time.

3. Training and practice time to conduct a peer assessment is important. Before the students divided into their groups, I showed them a sample role play tape and asked them to practice completing a peer assessment. This provided them the opportunity to become familiar with the observer sheet and to ask questions.

4. Groups of four are helpful. Because of class size, I divided some groups into four. Students reported that it was helpful to have one person operate the equipment and another person to observe and complete the observer sheet.

5. Students prefer having the role play scenarios provided to them rather than constructing scenes themselves. The first time I made this assignment, I asked the students to design their own situation to role play. In their evaluations of the project, they asked for the scenarios to be given to them. Subsequently, I wrote scenarios for them based upon my own life experiences.

6. Sufficient time is needed to complete the assignment. Students took approximately 2.5 hours to complete the project. They related finding the critique portion of the assignment very helpful, and they wanted plenty of time for this discussion. In some cases, they asked my opinion about the role play, and I found it helpful to be available to them.

7. Incorporating the role play self-assessment as part of the grade reduces some student anxiety. Students reported that my grading them on their self-assessments as well as on their rote play eliminated some of their apprehension about being recorded. Since my objective was for students to learn what is good communication and listening skills, I was comfortable incorporating the self-assessment as part of the grade.

8. Role play tapings should be reviewed by the faculty member. I examined the tape of each role play along with the self-assessment and observer sheet in order to ensure an accurate grade for each student. I made several written comments of my own, marked the grade, and returned the observer sheets to the students.

References

Airisian, P. W. (1996). Assessment in the classroom. New York: McGraw-Hill.

Bangert, A. (2003). An exploratory study of the effects of peer assessment activities on student motivational variables that impact learning. Journal of Student Centered Learning, 1, (2), 69-76.

Bernstein, J., Scheerhorn, S., & Ritter, S. (2002). Using simulations and collaborative teaching to enhance introductory courses. College Teaching, 50, (I), 9-64.

Collier, K. (1998). Once more with feeling--identification, representation and the affective aspects of role play in experience-based education. In J. Rolfe,

D. Saunders, & T. Powell (Eds.), The internal simulation and gaming research yearbook (vol. 6, pp. 145-153). London: Kogan Page.

Collier, K. (2000). Dramatic changes--a new action model for role-play practice. In J. Rolfe, D. Saunders, & T. Powell (Eds.), The internal simulation and gaming research yearbook (vol. 8, pp. 47-58). London: Kogan Page..

Earl, S. (1986). Staff and peer assessment of a group project designed to promote skills of capability. Programmed Learning and Educational Technology, 25, 327-339.

Falchikov, N. (1986). Product comparisons and process benefits of collaborative peer group and self-assessment. Assessment and Evaluation in Higher Education, 11, 146-166.

Haaga, D. (1993). Peer review of term papers in graduate psychology courses. Teaching Psychology, 20, 28-32.

Mowl, G., & Pain, R. (1995). Using self and peer assessment to improve students' essay writing: A case study from geography. Innovations in Education and Training International, 32, 324-335.

Marcoulides, G., & Simkin, M. (1991). Evaluating student papers: The case for peer review. Journal of Educational Business, 167, 80-83.

Taylor, J., & Wafford, R. (1978). Learning and the simulation game. United Kingdom: Open University Press.

Topping, K. & Ehly, W. (1998). Introduction to peer assisted learning. In K. Topping & W. Ehly (Eds.), Peer assisted learning. Mahway, NJ: Erlbaum Publishers.

Van Ments, M. (1983). The effective use of role-play: A handbook for teachers and trainers. London: Kogan Page.

Marilyn Lockhart, Montana State University 

Marilyn Lockhart, Ed. D., is an Associate Professor in the Department of Education