Library instruction: online or in the classroom?

Abstract

Three groups of students in an English composition course received basic information literacy instruction by completing an online tutorial, attending a presentation by a librarian, or doing both. An analysis of pre and post-test scores revealed that while learning occurred in all three groups, there was a significant difference in learning between the group who received both forms of instruction and the group who attended an in-class presentation. While most students were comfortable with online learning, almost half still preferred classroom instruction.

Introduction

Kraemer Family Library at the University of Colorado at Colorado Springs (UCCS) introduced its first Web-based information literacy skills tutorial in the spring of 2004. Called LiONiL (Library Instruction Online for Information Literacy), the tutorial provides a program of instruction that emphasizes basic library concepts and skills that students can apply in all disciplines. LiONiL was originally designed as an alternative to in-class instruction in English 141, Writing and Rhetoric II, a core composition course that introduces students to the reading and writing activities that are integral to academic research. LiONiL raises students to a common level upon which librarians and faculty can later build more information literacy competencies unique to a particular course or discipline. The tutorial also provides faculty with a mechanism to help satisfy the information literacy core education requirement at the University and a means for measuring learning outcomes for course and discipline assessment plans.

A statistical analysis of the LiONiL pre and post-test scores was conducted after the inaugural semester in spring 2004 to measure the efficacy of the tutorial. The results of the study (Byerley, 2004) demonstrated that significant learning occurred, although scores on the post-test were not as high as hoped. The Student Satisfaction Questionnaire at the end of LiONiL revealed that a small majority of students (54 percent) preferred the online format of instruction because it was flexible, self-paced, interactive, and convenient. Even still, almost half of the students in the sample remained skeptical of online learning. These students were more comfortable in a live setting where the librarian could explain concepts, demonstrate skills, and answer questions. While the study demonstrated that online instruction was effective, it also concluded that more evidence was needed to determine if LiONiL is a viable alternative to classroom instruction.

Review of the Literature

Other studies have compared online library instruction and face-to-face library instruction with varying results. At the University of North Carolina at Chapel Hills (Holman, 2000) librarians found that online instruction was equally as effective as traditional instruction. This study also revealed that both groups of students felt confident after instruction although their confidence did not correspond to their posttest scores. Librarians at SUNY Albany (Germain, 2000) and SUNY Oswego (Nichols, 2003) also found that students learn as much with online instruction as they do with in-class instruction and are satisfied with the online format.

On the other hand, a study conducted at Deakin University in Australia (Churhovich, 2002) revealed that classroom instruction was more effective. Librarians examined a group of students who received traditional library instruction, a group who completed a tutorial independently, and a group who completed the tutorial with assistance from a librarian. The analysis of post-test scores revealed that students who had classroom instruction performed better and felt more confident about using library skills than students who completed an online tutorial with or without assistance.

Most recently a study at Auburn University demonstrated that their online program was effective but needed additional improvement (Noe, 2004). This study did not compare students who completed an online tutorial with students who received in-class instruction. Like the Auburn study, the UCCS study did not compare learning outcomes of students who completed the tutorial with those of students who received face-to-face instruction.

Methodology

During the 2004-2005 academic year the author conducted a second study to compare the learning outcomes of students who completed LiONiL with students who received face-to-face instruction. The author also studied a third group of students who were exposed to both teaching methods. Writing instructors assigned their English 141 sections to one of three groups: one that would complete LiONiL, one that would attend a seventy-five minute presentation by a librarian, and one that would complete LiONiL and then attend a presentation. The presentation closely followed the pedagogy in the online tutorial, varying only in that it introduced students to three online periodical databases while the tutorial introduced only one.

Students in all three groups completed a twenty question pre-test near the beginning of the semester, and an identical post-test during the last two weeks of the semester after all library assignments were turned in. At the beginning of each test students were asked to rate their level of confidence on eleven information literacy concepts and/or skills. Students in Groups A and C were also asked to complete a Student Satisfaction Questionnaire about LiONiL. The questions elicited feedback about the content, interface, and length of the tutorial, as well as overall satisfaction with the online program as a learning tool.

Results

Twelve sections of English 141 with approximately 20 students in each participated in the study. However many students in each section either did not complete the pre or post-test or did not sign the student consent form required by the UCCS Institutional Review Board giving permission to use their scores and responses in the study. A total of 141 students, 82 percent of whom were freshmen or sophomores participated in one of three groups which will henceforth be referred to as Group A (N=53) who completed LiONiL; Group B (N=43) who received in-class library instruction; and Group C (N=45) who completed LiONiL and then received in-class instruction.

A repeated measures analysis of variance (ANOVA) was conducted to compare the pre and post-test scores of all three groups. There was a significant main effect for time, Wilks' Lambda =.46, F(1,138)= 159.55, p<.0005, and the effect size was large (eta squared = .54) indicating that there was a significant change in scores from the pre-test to the post-test in each group. There was not a significant interaction effect, Wilks' Lambda =.99, F(2,138)=.34, p>0.71, and the effect size was small (eta squared =.005) showing that no one group changed more than the other two.

However, the between subjects effect was .039 with a small effect size (eta squared =.05). Post-hoc comparisons using the Tukey HSD revealed that the mean score for Group C (M=14.04, SD=1.82) was significantly different from Group B (M=12.79, SD=2.69). The change in scores for students receiving both forms of instruction was significantly different from the group receiving classroom instruction, suggesting that the addition of LiONiL had an influence on the higher test scores in Group C.

While all three groups did better on the post-test, none did as well as hoped. Post-test scores ranged from 7 to 18 for Group A, 6 to 18 for Group Band 9 to 18 for Group C. Two possible explanations for the low post-test scores come to mind. First, instructors did not assign a grade to the post-test which may have had an effect on student motivation. If the post-test had been graded on a percentage basis, about half of the students in Groups A and B and one third of students in Group C would likely have failed the test with scores of 65 or less. Second, the tests were hand graded and students did not receive their pre-test scores before instruction took place. Had students been able to gage their pre-test knowledge, they likely would have approached the instruction with more intent to learn.

Group C did best overall increasing their mean score from 11.27 to 14.04. A look at individual scores revealed that fifty three percent improved their scores by three or more points, 77 percent improved their scores by two or more points, and 86 percent improved their scores by at least one point.

Group A did next best, increasing their mean score from 10.98 to 13.36. Forty-seven percent improved their scores by three or more points, 66 percent improved their scores by two or more points, and 75 percent improved their scores by at least one point.

Group B came in last, increasing their mean score from 10.26 points to 12.79 points. Fifty-four percent improved their scores by three or more points, 73 percent improved their scores by two or more points, and 82 percent improved their scores by at least one point. Group B showed more improvement than Group A, however, their mean score on the pre-test was almost a point lower and that may account for the difference.

At the beginning of the pre and post-tests students were asked to rate their confidence on a scale of one to five, one indicating no confidence and five indicating complete confidence for the following concepts and skills:

1. Understanding the kinds of information in the library.

2. Identifying main concepts in a thesis statement.

3. Searching the Kraemer Online Catalog.

4. Finding information on the World Wide Web using search engines such as Google and Yahoo.

5. Selecting and using periodical indexing and abstracting services.

6. Understanding the advantages of using electronic library resources over Internet search engines.

7. Identifying scholarly or peer-reviewed journals.

8. Differentiating between primary and secondary sources.

9. Retrieving print and electronic sources in Kraemer Family Library.

10. Connecting to online resources at the Library from off campus.

11. Using information legally and ethically.

A paired samples t-test was used to analyze the pre and post-test confidence ratings. Each group gained more confidence after instruction. As with the post-test scores Group C once again had the highest mean post-test confidence rating (4.08) followed by Group A (4.03), and Group B (3.92). One possible explanation for this is that the tutorial provides consistent repeatable hands-on instruction. A presentation, on the other hand, is less structured since the librarian must respond to all levels of experience that students bring to a session. Learners may ask for further clarification in a class setting, but often they are afraid to speak out. Also there is no opportunity to repeat face-to-face instruction. It must also be pointed out that the post-test confidence ratings were not correlated to the post-test scores. In fact, a quick examination of those students who answered thirteen or fewer questions correctly on the post-test revealed that despite their poor performance, more than half gave themselves a confidence rating between 4 and 5. While this trend is somewhat disturbing, it is hoped that their increased confidence will encourage them to use the library and seek additional assistance.

When the skills were ranked by most improvement to least improvement the rankings were consistent for each group even though the rates of change were different. Everyone achieved the greatest increase in confidence on skills 6 and 7, understanding the advantages of using Web-based library resources, and identifying scholarly or peer reviewed journals, followed by skills 3 and 5, searching the Kraemer online catalog and selecting and using periodical indexing and abstracting services. Predictably, all three groups felt very confident before and after instruction on skill 4, finding information on the WWW using search engines such as Google and Yahoo. They also expressed a high level of confidence before and after instruction on skill 11, using information legally and effectively.

Students in Groups A and C were asked to submit feedback on the LiONiL Student Satisfaction Questionnaire. Fifty three students in Group A and 45 students in Group completed the questionnaire. Responses were not correlated to their test scores or confidence ratings, but the feedback helped librarians gain more perspective on student learning styles and preferences. Most students were satisfied with the tutorial. Sixty four percent of Group A and 87 percent of Group C believed the tutorial helped them learn new research skills. Seventy-seven percent of Group A and 96 percent of Group C believed that LiONiL increased their awareness of resources and services available through the Library. When asked if LiONiL was an effective format for information literacy instruction 74 percent of Group A and 85 percent of Group C agreed that it was.

However, when asked which method of instruction they preferred, 53 percent of the students in Group A selected the online option. This figure is almost identical to the percentage of students (54 percent) that preferred the online format in the earlier study. The results for Group C, however, were somewhat surprising. When asked which mode of instruction they preferred, only 36 percent of these students opted for the tutorial. Two students in Group A commented that both forms of instruction would have been valuable. Two students in Group C stated that they liked both formats because the tutorial laid out the basics and then they were able to ask questions in the class instruction session.

The reasons cited for student preferences were much the same as in the earlier study. Almost every student who favored online instruction did so because they preferred learning at their own pace and enjoyed the convenience and flexibility that LiONiL offered. In addition students appreciated the active learning components that enabled them to practice skills as they learned them and the interactive quizzes that provided instant feedback. Some students thought it made sense to learn library research skills online since the Web environment is where most of their research is conducted. Others liked the fact that the tutorial did not take up class time. Students who favored class instruction felt that interaction with a person is preferable to interaction with a computer. Most stated that learning is easier for them when concepts and skills are demonstrated and explained in person. Furthermore, the librarian is present to answer questions. A few students admitted that they learn better in class where they are forced to pay attention, while others confessed that they simply do not like to sit in front of a computer for long periods of time.

Conclusion

Online tutorials cannot completely substitute for human connection (Dewald, 1999). This small study demonstrates that while LiONiL is an effective model for teaching information literacy skills, there are many students who because of their learning styles still prefer the social context of the classroom. Librarians at Kraemer initially thought that LiONiL could be integrated into course instruction as an alternative to a class presentation. Now they regard LiONiL as a learning tool that is most effective when assigned as a prerequisite to a class presentation. LiONiL will continue to be used in distance education programs and in courses where class instruction is not an option.

As a result of the study, the Writing Program at UCCS has formally integrated LiONiL into the curriculums of English 141, and two other courses, Business and Administrative Writing, and Technical Writing and Presentation. Most instructors are also scheduling follow-up instruction by a librarian. The pre and posts-tests have been migrated to a course management software program that will automatically score the pre and post-tests and provide an easier mechanism for tracking assessment data. It is hoped that the instant feedback on the pre-test will increase student awareness of the importance of library instruction.

LiONiL won the UCCS Innovations in Teaching with Technology Award in 2005 and was a runner-up for the Colorado University President's Faculty Award for Advancing Teaching and Learning through Technology. Professors in several disciplines have expressed a desire to use the tutorial in their courses and assessment strategies and are recognizing the importance of information literacy. The Beth-El College of Nursing and Health Sciences has provided funding for a nursing tutorial which is currently under development. Following its completion librarians will create an online tutorial for the College of Education.

References

Byerley, Suzanne, L. (2004). Teaching Basic Information Literacy Skills Online. Academic Exchange Quarterly, 8(4), 257-61.

Churchovich, M. and Oughtred, C. (2002). Can an online tutorial pass the test for library instruction: an evaluation and comparison of library skills instruction methods for first year students at Deakin University. Australian Academic and Research Libraries, 33(1), 343-9.

DeWald, N. (1999). Transporting good library instruction practices into the web environment: an analysis of online tutorials. Journal of Academic Librarianship, 25(1), 26-32.

Germain, C., Jacobson, T., and Kaczor, S. (2000). A comparison of the effectiveness of presentation formats for instruction: teaching first-year students. College & Research Libraries, 61(1), 65-72.

Holman, L. (2000). A comparison of computer-assisted instruction and classroom bibliographic instruction. Reference & User Services Quarterly, 40(1), 53-60.

Nichols, J., Shaffer, B. and Shockey, K. (2003). Changing the face of instruction: is online or in-class more effective? College & Research Libraries, 64(5), 378-88.

Suzanne L. Byerley, University of Colorado at Colorado Springs

Byerley, M.L.S., is an Assistant Professor and the Electronic Reference Services Librarian at Kraemer Family Library