New assessment technology: the PRS alternative.

Abstract

Personal Response Systems (PRS) can be used for instant, interactive assessment during library instruction sessions. The technology is a good active learning tool for today's freshmen, who are at the dualistic stage of cognitive development and enjoy using technology to learn. The benefits and barriers of PRS are discussed within the context of library instruction.

Introduction

Administering a one-shot library instruction session on first-year college students is a challenge. Personal Response System (PRS) technology allows a new form of active learning to take place that simultaneously yields immediate assessment data to the librarian. Students use it to answer multiple choice questions anonymously via handheld devices, and the immediate results allow for better customization of the library instruction. This article discusses previous active learning methods used in library instruction and explains why PRS is an exciting new active learning tool for today's technologically savvy freshmen. It also describes the hardware and software and discusses the benefits and challenges of using PRS to enhance library instruction.

Active Learning in Library Instruction

The concept of "active learning" is not new to the field of librarianship. Drueke, Allen and Lorenzen have all documented the rise of the active learing movement. [1] When trying to define active learning, many researchers cite Bonwell and Eisen, who in their 1991 report gave several descriptive characteristics now considered definitive. With active learning, students must do more than merely sit in class and listen to the instructor teach: instructors place less emphasis on the mere transmission of information and more on developing students' skills. Students are involved in higher order thinking processes like analysis, synthesis and evaluation while performing activities such as reading, discussion, and writing. Active learning places more emphasis on students' exploration of their own attitudes and values and requires both physical and psychological energy. [2]

Some question if there is a difference between learning and active learning, and posit that all learning is by definition active. There is, however, evidence that students learn more when they are actively engaged in a classroom setting, as opposed to merely passively sitting and listening to a lecture. Students dislike the lecture format because it bores them, and they do not retain much from the lecture beyond the first few minutes. [3] Bonwell also notes that the lecture is "less effective in promoting thought or in changing attitudes." [4] The literature provides the following examples of active learning for teaching faculty and librarians. Bonwell, speaking in general terms, writes about the modified lecture, tests and quizzes, classroom discussion, writing in class, and case studies. [5] Ragains used the "jigsaw method" of classroom discussion in his library research classes at Montana State University, Bozeman, and found through informal assessment that students enjoyed these sessions better and that it inspired more commitment to library instruction from the teaching faculty involved.[6] Mahry wrote about "cooperative learning" techniques she used at the University of Minnesota, in which she recognized that if she were willing to "give up some control, students will learn more and retain that knowledge longer." [7] Krajewski and Piroli, at Simmons College, learned to combine their old instructional styles with new elements of active learning, and created successful and innovative self guided tours and a game of "library jeopardy" for their students. [8]

Active Learning for Freshmen

There is evidence that some active learning library instruction experiments were not entirely successful because students had trouble seeing the relationship between the abstract "active learning" exercise and the research skills needed to complete an assignment. [9] This is probably because an abstract exercise requires a student to think at an advanced developmental level they are not yet prepared for. According to research by Piaget (1952) and later by Perry (1968) and supported by more recent studies, first-year college students often rely on others, in particular authority figures, for answers. [10] Perry labeled this stage of development "dualism," characterized by an absolute view that facts are equivalent to knowledge. [11] Dualists are "literal-minded" and "want the "right answer." [12] They have a desire for structure, and so would benefit from active learning that is integrated more closely into the in-class session. Non-graded, or formative, quizzes interspersed throughout the lecture offer all the benefits of active learning without losing sight of its connection to the lecture. Therefore, quizzes offer the perfect active learning vehicle for the dualist, testing for factual knowledge rather than directing them towards a seemingly unconnected 'think' exercise.

College students have changed since many of these research studies were published. Although still starting college at the dualistic developmental stage, the majority of freshmen are now part of Generation Y, born in or after 1982 and also known as the 'Net Generation or the Digital Generation due to their technological prowess. [13] Active learning needs to be modified to accommodate the characteristics of this new generation. Generation Y sees technology not as a tool, but as something they are organically connected to, [14] as well as something they rely on for learning. [15] Growing up in the digital age has meant that learning, entertainment, and instant gratification are all inextricably linked. [16] Thus, it makes sense to assimilate information technology into learning. As James Fitch noted while discussing the implementation of new technology to assist teaching at Auburn University, "perhaps technology can create new avenues of interaction that will meet the needs of a changing student population." [17] For Generation Y, information technology has replaced the traditional reference librarian when it comes to doing research; they prefer to use the Internet rather than asking a reference librarian. [18] Therefore, using information technology to implement an active learning exercise that is at first-year college students' level of cognitive development seems like a positive step.

PRS

Librarians at the University of Southern California's Leavey Library use PRS in their information literacy program for instant, interactive assessment in combination with a lecture. Also called "clicker technology," it became known when used for audience polling on the game show "Who Wants to Be a Millionaire." Students use remote control-like "clickers" similar to those on the television show to answer multiple-choice questions anonymously, and they get the breakdown of class responses instantly. The hardware we use is manufactured by GTCO Calcomp, [19] and this is used in conjunction with Turning Point software. Other, similar technologies include Einstruction's Classroom Performance System (CPS) [20] and Hyper-Interactive Teaching Technology (H-ITT)[21]. The software allows the instructor to create a PowerPoint presentation of multiple-choice questions testing the key concepts in their lecture. When students punch in their answers, infrared light transmits the information to a receiver in the wall and you can display the results. Sessions can be exported to an Excel file and saved for future analysis, if desired. We registered all clickers by number ahead of time so that, even though the individual's identity remains anonymous, we know how well they did across all questions.

Leavey librarians currently use PRS in their library instruction sessions for Writing 140, a required class for most freshmen. We use it for pre and post testing by asking the same set of four multiple choice questions testing targeted outcomes from ACRL's Information Literacy Competency Standards. [22] This allows us to initially gauge student knowledge so we can focus the session more on what they don't know. In addition, it allows us to see how well they understood our lecture after the fact. PRS can also be used intermittently throughout the lecture by checking to make sure a concept is understood before moving on. Previously we used a paper version of the same questions to pre and post test first-year students, but we didn't get the same positive reaction we get with this tool.

Benefits of PRS

PRS specifically benefits our WRIT 140 first-year students by using information technology, which they enjoy, to assess their comprehension in a dualistic, right/wrong fashion that is appropriate to their level of cognitive development. It allows the library instructor to immediately perceive knowledge gaps in the class and evaluate how well their lecture has filled these gaps, which is extremely valuable. PRS has many other, more general benefits for students. It incorporates active learning for each student in a non-threatening, anonymous way: students invest in answers at the outset which they have a stake in for the rest of the lecture. As Professor Mazur, a Harvard professor, noted when he used it in his classroom, "something happens pedagogically when a person takes a position, commits to it, and is forced to defend or re-evaluate it that just does not happen during an ordinary lecture." [23] Also, by centering questions around the crucial concepts you wish to convey, it reinforces what is important. PRS caters to students with learning styles other than just auditory.

In our instruction sessions, our dualistic-minded students often ask us what the "right answer" is when they do the pre-test, but we don't reveal answers until after the lecture and students have re-taken the test. This builds curiosity and encourages classroom discussion. As we administer each question in the post-test and reveal the breakdown of responses, we ask someone who chose the correct answer to explain their choice. This facilitates peer learning while simultaneously providing the instructor with valuable insight into how students integrate new knowledge. PRS allows for non-confrontational competition to get the "right" answer, and promotes a sense of community among the class and pride in class achievement when everyone selects the correct answer. Although we have not formally assessed PRS' effect on how much students learn, preliminary experimentation seems to confirm they at least like it and pay attention more than in library sessions where it is not used. At Auburn University, instructors in the Department of Communication Disorders piloted a similar technology called LearnStar in two communications classes and found that students had a positive reaction when polled and that instructors felt it increased student attentiveness in class. [24] At the University of Arkansas, physics professor Paul Thibado used PRS in a large physics lecture class and found that it increased attendance by almost 50% and increased the average grade in the class by 28%. [25]

Shortcomings of PRS

Undoubtedly, there will be barriers to using PRS in library instruction. One of the main problems is the cost. The hardware, its installation, the software, and ongoing maintenance costs are all required to implement this technology. At USC, we paid a little over $3k for implementing PRS in one of the learning rooms, which included the two PRS receivers, their installation, and clickers. We used a pre-existing laptop, and a TurningPoint license for a 50-seat classroom is roughly another thousand dollars. You can look for grants to fund the project, or partners within your institution with similar instructional goals. At USC, Leavey Library partnered with the Center for Scholarly Technology (CST) to reduce the financial burden. When LearnStar was installed at Auburn, it was funded by a grant from the Auburn University College of Liberal Arts. [26] A multi-faceted concern that librarians may have in implementing a program like this is time. When using PRS in your instructional program, you will need additional prep time. Though relatively easy to learn, it does take time at the outset to become acquainted and comfortable with the software, but fortunately this is a one-time "expense." You must also decide how to use the technology in the classroom--as a pre/post test, or a check for comprehension throughout the session? Either way, you must set up the questions beforehand, but the good news is that if you teach similar sessions again and again, you can use the same files, thus diminishing the prep time.

Another concern is that you will be giving up some valuable class time in order to implement PRS in your sessions. In our experience we have found that this kind of activity can take up to 10 minutes, which is not inconsiderable in a 50 minute session. There are ways, though, to compensate for this loss of time. Online tutorials can be efficient tools that complement the materials taught in class. They can be completed by the students prior to coming to the session, which creates more class time. Some may be wary of PRS, but there are ways to make them feel more comfortable with it. You can set up hands-on training sessions for them where you guide them through each step, and you can even offer to team-teach their classes until they feel more comfortable. It also helps to surround yourself with others who use it. At USC, CST has set up a "community of practice"[27] for those Faculty members interested in or already utilizing PRS. [28] During the Fall 2004 semester, they held monthly luncheon meetings in which members could exchange ideas, brainstorm for new uses and simply engage in discussion about how things were working in their classes.

Conclusion

The purpose of this article was to introduce the application of Personal Response System technology in the library setting. The advantages and disadvantages were described and analyzed. Initial utilization shows promise in promoting active learning in library instruction to first year college students. In the future, the authors hope to investigate the greater impact of PRS technology on student learning and collect formal feedback from the teaching faculty about its effectiveness. In the meantime, the authors hope this article encourages more librarians to begin experimenting with interactive technology in the classroom.

Endnotes

[1] Jeanetta Drueke, "Active Learning in the University Library Instruction Classroom," Research Strategies 10, 2 (1992): 77-83; Eileen E. Allen, "Active Learning and Teaching: Improving Postsecondary Library Instruction," The Reference Librarian 51/52 (1995): 89-103; Michael Lorenzen, "Active Learning and Library Instruction," Illinois Libraries 83, 2 (2001): 19-24.

[2] Charles C. Bonwell and James A. Eisen. 1991. Active Learning: Creating Excitement in the Classroom. ASHE-ERIC Higher Education Report No. 1 Washington, D.C.: The George Washington University, School of Education and Human Development. 2-3.

[3] Ibid., 8-9.

[4] Ibid., 9.

[5] Ibid.

[6] Patrick Ragains, "Four Variations on Drueke's Active Learning Paradigm," Research Strategies 13, 1 (1995): 40-50.

[7] Celia Hales Mabry, "Using Cooperative Learning Principles in BI,'" Research Strategies 13, 3 (1995): 183.

[8] Patricia R. Krajewski and Vivienne B. Piroli, "Something Old, Something New, Something Borrowed, Something Blue: Active Learning in the Classroom," Journal of Library Administration 36,1/2 (2002): 177-194.

[9] Karen Williams and Jennifer Cox, "Active Learning in Action," RQ 31, 3 (1992): 326-331.

[10] Jean Piaget, The Origins of Intelligence in Children. (New York: International University Press, 1952); William G. Perry, Forms of Intellectual and Ethical Development in the College Years: A Scheme (New York: Holt, Rineheart & Winston, 1968); Nancy J. Evans, Deanna S. Forney and Florence Guido-DiBrito, Student Development in College: Theory, Research, and Practice (San Francisco: Jossey-Bass Publishers, 1998).

[11] Evans et al, 131.

[12] Elizabeth J. McNeer, "Learning Theories and Library Instruction," Journal of Academic Librarianship 17, 5 (1991): 295.

[13] Neil Howe and William Strauss, Millennials Rising: the Next Great Generation (New York: Vintage Books, 2000).

[14] Stephen R. Merritt, "Generation Y: a Perspective on America's Next Generation and Their Impact on Higher Education," reported by Shelley Neville, Serials Librarian 42, 1/2 (2002): 46.

[15] Diana Oblinger, "Boomers, Gen-Xers & Millennials: Understanding the New Students," EDUCAUSE Review 38, 4 (July/August 2003): 37-47.

[16] Ricigliano, Loft, "After X Comes Y: Teaching the Next Generation," in Library User Education in the New Millennium, ed. Julia K. Nims and Ann Andrew, 124-125. (Ann Arbor: Pierian Press, 2001).

[17] James L. Fitch, "Student Feedback in the College Classroom: A Technology Solution," Educational Technology, Research and Development 52, 1 (2004): 77.

[18] Oblinger; Susan Gardner and Susanna Eng, "What Students Want: Generation Y and the Changing Function of the Academic Library," Portal: Libraries and the Academy (projected July 2005)

[19] GTCO CalComp, "InterWrite Products," http://www.gtcocalcomp.com/interwriteprs.htm (accessed 1/26/05).

[20] Einstruction, "Classroom Performance System," http://www.einstruction.com/ (accessed 3/23/05).

[21] Hyper-Interactive Teaching Technology, "http://www.h-itt.com/ (accessed 3/23/05).

[22] Association of College and Research Libraries, "Information Literacy Competency Standards for Higher Education," American Library Association (2000), http://www.ala.org/ala/acrl/acrlstandards/ informationliteracycompetency.htm(accessed 1/19/05).

[23] Eric Goldscheider, "In Class, Gadgets for Interacting, and No One Need Nod Off," New York Times, November 12, 2000.

[24] Fitch, "Student Feedback"

[25] Elisabeth Butler, "Who Wants to Be a Physicist?," Arkansas Business 17, 46 (Nov 13, 2000): 13.

[26] Fitch, 77.

[27] See Etienne Wenger, Communities of Practice: Learning, Meaning, and Identity (New York: Cambridge University Press, 1998).

[28] The Center for Scholarly Technology, Teaching and Learning Services, "Theme: Networked Interaction & Classroom Engagement With Personal Response Systems," University of Southern California, http://www.usc.edu/isd/locations/cst/tlsnew/currentthemes/nice.html (accessed 1/12/05).

Susanna Eng, University of Southern California

Susan Gardner, University of Southern California

Eng is Assistant Instructional Services Coordinator and Reference Librarian. Gardner is Collection Development Coordinator and Reference/Instruction Librarian in USC's Leavey Library.