A comparison of lecture ratings by Native Speakers of English with EFL students at two universities: University of Mississippi and Dokuz Eylul University.

Evaluations by students of the University of Mississippi were compared to the evaluations of students at Dokuz Eylul University related to specific lectures on communication skills. The students of the University of Mississippi were Native Speakers of English (NSE) while the students at Dokuz Eylul University spoke English as a Foreign Language (EFL). A set of lectures were developed over a ten-year period by using a continuous improvement feedback loop between student and instructor. Once the feedback data stabilized with undergraduate NSE, a comparison was made with undergraduate EFL students. Also undergraduate EFL students were compared with graduate NSE. The evaluations focused on the likeability of the lecture. Each student completed a feedback sheet indicating his or her reaction to each lecture. The results showed a remarkable similarity in response trends. If the response to a specific lecture was determined high by NSE, it was likely to be rated high by EFL students and vice versa.

**********

Using a continuous improvement feedback loop between student and instructor is an effective way to get valuable information for improving instruction (Blackbourn, Payne, & Hamson, 1997). A simple system of having students deposit a short feedback sheet at the end of each class session provides information to consider for improvement as well as an objective measure for determining individual class performance (Payne & Blackbourn, 1995, Payne & Blackbourn, 1996).

Using feedback from students, over time, it has been demonstrated that the reaction to individual lectures tend to stabilize. By assessing each class presentation, it becomes fairly easy to determine that some lectures are better than others. It is common knowledge that university students, class by class, vary in their reactions to the same course, semester after semester, just as individual audiences of a theatrical performance react differently to the same song, number, or joke, performance after performance. In other words, some audiences respond more favorably than others; however, trends exist over time regarding a specific song, number, or joke.

Performances on stage and in the theatre gauge how well things are going from the audience's appropriate laughter and applause. In a university course where reaction to the teaching/learning process is to be assessed, applause and excessive laughter, for the most part, are inappropriate. However, a straightforward, short, concise feedback form handed in at the end of each class session is appropriate and helpful in determining student reaction.

A form containing a simple scale of 0-100 (zero representing poor, bad, worst, while one hundred represents outstanding, best, exemplary) and space for an opportunity to give suggestions or make specific comments was used. Students were instructed to fill out the form at the end of each class and place it in a box that was located by the door as they exited. The form used in this study was developed over a ten-year period.

A detailed account of the procedure and an illustration of the form used may be found in the Proceedings of the Eleventh American University in Cairo Conference (Payne, 2004) and in the Journal of Instructional Psychology, Improving Classroom Instruction Through 'Best-of-Class' Techniques (Blackbourn, Payne, Burnham, Elrod, & Thomas, 2000).

During the ten-year developmental period, each lecture was perfected and suggestions from the feedback of students were taken into consideration. Overtime, the data began to stabilize and trends became apparent that certain class presentations scored higher while others scored lower regardless of the semester. This study was designed to determine if these trends would hold and transcend a culture where English is practiced as a Foreign Language.

Through the award of a Fulbright Grant, the lectures were presented to undergraduate students at Dokuz Eylul University in Izmir, Turkey in the Department of American Studies. A comparison of the results of senior year Turkish students with undergraduate students made up of sophomores, juniors, and seniors at the University of Mississippi, may be found in Figure 1, and a comparison of two classes, one of freshmen and the other of sophomores at Dokuz Eylul University with a first year graduate class of students at the University of Mississippi, may be found in Figure 2.

[FIGURES 1-2 OMITTED]

During the Fall semester of 2003, ten sessions on basic communication skills were presented to a group of forty-nine seniors at Dokuz Eylul University. Each class session was primarily lecture supplemented by short video clips. Each session was composed of two, forty-five minute segments separated by a fifteen minute break. The average ratings provided by the students were compared with the same ten sessions presented to twenty-seven undergraduate students made up of sophomores, juniors, and seniors during the Spring semester of 2003, at the University of Mississippi. Figure I shows the EFL-students at Dokuz Eylul University rated the course 95.0 while the NSE at the University of Mississippi rated the course 95.5. The ratings for each lecture are similar with the exception of session one. By ignoring or omitting session one and observing sessions two through ten, the similarity of each session's response, by each group of students, is remarkably similar. The reason for the discrepancy found in session one between the two groups is unknown and is open for speculation.

During the Spring semester of 2004, nine sessions on advanced communication skills were presented to two groups of students at Dokuz Eylul University. One class was composed of fifty-three freshmen taught in the morning and the other class was forty-nine sophomores taught in the afternoon. The format was the same as that used with the seniors but the content was slightly more advanced. The ratings of the two groups were compared with the same nine sessions presented to twenty-six first year graduate students at the University of Mississippi. Figure 2 displays the data of the three groups.

The EFL freshmen rated the course the highest with a 99.2 while the EFL sophomores rated it 97.2. The NSE graduate students rated it 96.2. It is interesting to note that both EFL groups showed similar ratings for each of the nine sessions resulting in a parallel ascending trend from session one to nine. The NSE graduate students responded higher to each session with the exception of session nine where the freshmen outscored both groups with a 99.5. Also the NSE graduate students ranked sessions one through five significantly higher than the two EFL groups. In this particular comparison it appears the graduate students rated the first half of the course higher, possibly because they were somewhat more prepared. But by the sixth and seventh sessions, both undergraduate classes caught up, or at least began to rate the sessions similarly.

In conclusion, EFL students tended to rate the lectures similarly to NSE classes. Graduate students tended to rate the course higher in the first half of the semester but both undergraduate classes began to rate the sessions similarly to the graduate students during the last half of the semester.

One final note: It is important to reiterate that the development of the lectures was perfected over a ten-year period. But once perfected it does appear if the lecture is rated high by NSE it will also be rated high by EFL students and vice versa. Possibly, good teaching as well as mediocre teaching transcends cultures and languages.

References

Blackbourn, J. M., Payne, J. S., Burnham, S., Elrod, F., & Thomas, C. (2000). Improving classroom instruction through "best-of-class" techniques. Journal of Instructional Psychology, 27(1), 3-8.

Blackbourn, J. M., Payne, J. S., & Hamson, N. (1997). When the student is the customer. Record in Educational Leadership, 17(1), 37-39.

Payne, J. S. (2004). Using continuous improvement techniques to improve university instruction. Proceedings of the Eleventh American University in Cairo Conference. The American University in Cairo, Cairo, Egypt, 8-15.

Payne, J. S., & Blackbourn. J. M. (1995). Applying the TQM concept of continuous improvement to a TQM conference: How to make a very good conference even better. National Forum of Educational Administration and Supervision Journal, 12(2), 48-51.

Payne, J. S., & Blackbourn, J. M. (1996). TQM and schools of education: Creating responsive university-based learning environments. National Forum of Educational Administration and Supervision Journal, 13(1), 20-24.

Esim E. Payne, Associate Professor, Department of Curriculum and Instruction. James S. Payne, Professor, Department of Curriculum and Instruction, University of Mississippi.

Correspondence concerning this article should be addressed to Esim E. Payne, Associate Professor, Department of Curriculum and Instruction, The University of Mississippi, University, MS 38677.