A toy or a teaching tool? The use of audience-response systems in the classroom.

TELEVISION SHOWS SUCH AS "WHO WANTS TO be a Millionaire?" and "America's Funniest Home Videos" use audience-response systems. Members of the audience use handheld "clickers" to register their opinions or vote. This technology is now moving to the classroom.

A number of companies are marketing handheld electronic response systems as teaching tools. But the question is, "Are handheld electronic response devices merely entertaining electronic gadgets (toys), or do they really enhance learning?"

The Department of Agricultural and Extension Education at North Carolina State University (NCSU) has been conducting research on audience-response systems for the past two years. In this article, we will describe the results of two recent research studies.

What are Audience-Response Systems, and How Are They Used?

Audience-response systems consist of several components:

* wireless handheld devices (one for each student)

* a receiving device that receives signals from the handheld devices (This unit is plugged into a computer or communicates with the computer using wireless technology.)

* a computer with the audience-response system software installed

There are several manufacturers of this type of technology. We used the Classroom Performance System from E-Instruction (www.einstruction.com) in our research.

In using the system, a teacher will typically pose from five to 10 multiple choice questions to the class. Questions can be used at the start or end of the class for review, or they can be interspersed throughout the class. The question is typically displayed using some type of projection device. The students respond to the question using their responders.

After all students have responded, the results are then displayed on the screen. The number of students who selected each choice is displayed along with the correct response. No individual responses are shown. Each student is able to determine if his/her response is correct. If the answer is correct, this reinforces the learning. If the answer is incorrect, the student can ask for clarification or figure out why the response was incorrect.

Teachers immediately can tell if the class as a whole has learned the material. If the majority of the class misses the question, the teacher may need to reteach the concept before proceeding to new material.

Questions are prepared in advance by the teacher. Photographs, formulas and other graphical material can be incorporated into the question. The handheld responders are numbered. It is helpful to assign the student the same responder each time. The reason for this is that the responses of each student are collected in a database that operates behind the scenes. At the end of the class, the teacher can review the database to determine which students understand the concepts and to identify students who may need additional help.

The Research

The researchers formulated two research questions regarding the use of audience-response systems. These two questions were:

1 Does the use of an audience-response system result in increased student learning as measured by higher test scores?

2 Do students like using audience-response systems?

The first research study was conducted at Southern Nash High School in Nash County, North Carolina. This school has three agriculture teachers who all teach the same class--agriscience applications. Because the teachers teach the same content and were willing to cooperate with NCSU, it was possible to implement a carefully designed research study in this school. Each teacher agreed to teach one instructional unit using the audience-response system and to teach another unit without the audience-response system.

For the instructional unit (the FFA) two classes served as the treatment groups, and the other class served as a comparison group. Then, during the second instructional unit (leadership development), the group roles were switched. Students in the "treatment" group used handheld electronic responders to receive feedback. Students in the control group received traditional verbal feedback. At the end of each instructional unit, an achievement test was administered to each group. The experimental design for this study is visually presented in Table 1.

Class one had an enrollment of 23 students, class two had an enrollment of 20 students, and class three had an enrollment of 18 students. A total of 61 students were involved in the study. Each of the students experienced both the audience-response system feedback and the traditional verbal-based feedback.

Six hours of training, facilitated by one of the researchers, took place to teach the instructors how to use the handheld audience-response system. The questions posed to the students were multiple choice type questions. The students in the treatment group would respond to the question using the handheld electronic responder. The correct answer would then be displayed along with a summary of the percentage of students who chose each response.

The students would immediately know how they did and were able to see the total responses of the class. All the data were grouped, and there was no way for the students to identify how other students had responded. Based upon the student responses, the instructor could discuss or clarify any issues that needed to be addressed. In the control group, the instructor also asked questions but called on individual students and provided them verbal feedback about the adequacy of their response.

The three class instructors developed the two achievement tests used in the study. The test items were based upon instructional objectives and competencies drawn from a statewide test bank maintained by the state department of public instruction. The tests were examined for both validity and reliability and were found to be reliable and valid.

The treatment group (handheld electronic responders) had a mean achievement score on the test of 89.98 on a 100-point scale. The comparison group (traditional verbal feedback) had a mean score of 84.41 on a 100-point maximum test. These data are presented in Table 2.

An independent sample t-test was performed on these data. The value of t was 2.835, which is statistically significant at the 0.005 probability level. This means there was a "real" difference in the two groups. Students who used the electronic handheld responders scored higher on the exam than students who did not use the audience-response system. This represents a difference of about half a grade. It appears that the use of an audience-response system increases student learning.

Student Attitudes

A random sample of seven students from each class participated in focus-group sessions. The student comments were recorded, transcribed and then analyzed. The following words were used the most by the students in describing the audience-response system: helped, easy, participate, fun, learn, pay attention and understand. The frequency of each utterance is found in Table 3.

This research was replicated at NCSU using two biotechnology courses, one animal science course and one agricultural economics class. The audience-response system was used in the classes for the spring semester but not in the fall semester. When the final grades in the courses were examined, three of the four spring semester classes (audience-response system) had higher grades than the fall classes (no audience-response system). The student comments about the audience-response system were overwhelmingly positive. Two typical comments were:

"It is a nice alternative to typical question asking in a class setting because it gets the entire class to answer the question anonymously with instant and graphical feedback. This is great for a no-pressure learning atmosphere."

"I enjoyed the responder system because it allowed me to interact directly with the instructor while not feeling singled out."

Conclusions

Using an audience-response system holds potential for enhancing student learning. The students like the system and think it makes class more interesting. They also believe it helps them learn the material better. Students who used the response system did score higher on achievement tests. The use of this type of technology holds promise as a teaching tool.

Gary Moore, Barry Croom and James Flowers are on the faculty at North Carolina State University. They can be contacted at [e-mail], barry_croom@ncsu. edu and [e-mail].

John Conoley is a faculty member at East Carolina University. He can be contacted at [e-mail].

Table 1
Experimental Design

Agriscience               Instructional          Instructional
Applications                  Unit 1:               Unit 2:
Class Number                FFA History           Leadership
                                                  Development

Class 1: Instructor 1   Traditional Feedback  Electronic Feedback

Class 2: Instructor 2   Electronic Feedback   Traditional Feedback

Class 3: Instructor 3   Electronic Feedback   Traditional Feedback

Table 2
Achievement Scores by Group

Group        N    M        SD     SE

Treatment    61   89.98   8.72    1.12

Comparison   61   84.41   12.62   1.62

Table 3
High School Student Focus Group Word Frequency Totals

Relevant Word      Frequency on a
or Word String   Per Utterance Basis

Helped              65 instances

Easy                51 instances

Participate         38 instances

Fun                 38 instances

Learn               25 instances

Pay Attention       17 instances

Understand          10 instances