All things being equal: classroom vs. web.

Abstract

Since the initial offering of an online library/information literacy course, the question has been whether this form of instruction has any advantages over the traditional classroom method in terms of improved learning outcomes. The present study looks at the final exam grades of over 1,000 students to determine whether any such difference is indicated.

Introduction

Louisiana State University (LSU) is Louisiana's largest university, with a student population that hovers at about 31,000. The Instruction Group within LSU Libraries bears the primary responsibility for library/information literacy instruction at LSU. With only nine librarian-instructors, the task of enlightening so many students not only to the effective use of library resources but also to the much larger process of information literacy is a decidedly formidable task. Though library/information literacy instruction at LSU takes many forms, such as online tutorials and special one-time-only classes, the only platform that offers students extended time with a librarian for instruction in a range of information resources is the LIS 1001 Research Methods and Materials course. LIS 1001 sessions are offered twice each fall and spring semester and once each summer. Currently, the course's maximum capacity is 362 students each session. Thus, in a full year, the greatest number of students that can be accommodated is 1,490. The course is never at maximum capacity due to dropouts. In the summer of 2001, with an eye toward increasing the reach of the library's instructional efforts, an online version of the LIS 1001 course was tested. Students who enrolled did not know at first that they would be taking the course online. These students actually met in a classroom each day at the advertised time.

The web-based course was designed as a pilot for a section that would be taught totally online in the following fall semester. The instructor assigned to the summer section explained this to the class and asked them to read the instructional materials online and complete any assigned tutorials before the class session. The students were told that, in return, they would receive help in doing the assignments during class time. Students were free to ask questions for clarification as they worked through each lesson. In this hybrid way and with access to immediate feedback from the students, the instructor and course designer were able to gauge the limitations of the proposed course plan and material, to discover whether the lessons and assessments were clear, and, ultimately, to improve the course so that it could be offered successfully without the necessary presence of an instructor in a classroom.

Since then, the online version of LIS 1001 has been refined each nine weeks it has been offered and seems to be popular with the students, as evidenced by its normally high enrollment rate. Online sections that began 100% full retained an average 90.8% of their students during the study period (summer 2001 to fall 2002), as compared to 89.2% retention for the classroom course during the same interval. In this respect, our experience with virtual instruction contradicts that reported by Hiltz in her 1997 study (n. pag.) and by Jorgensen in her 2002 survey of asynchronous (i.e., virtual) learning (5). These authors each pointed to higher drop-out rates in virtual, as opposed to classroom, classes as a concern of which online course planners should be aware. Even so, and regardless of the apparent popularity of our online course, a question has lingered since that first virtual class was offered. The question is this: Is the online delivery method more effective than the classroom method? Casual observation of the students in that first online class made it clear that those students were totally focused and engaged, albeit perforce. They had no choice but to concentrate, or they would never complete the lessons and the work on which they were being graded. In this, there was a clear contrast with the average classroom student, who might be prone to lose focus and become distracted during a lecture or demonstration. It seemed apparent that putting the full burden of learning on the student made for a much more assiduous student. But did this assiduousness translate into better learning?

Literature Review

Others have attempted to compare delivery platforms in varying contexts and with somewhat varying results. Most of these comparisons have not found any real difference in the effectiveness of one platform over the other. The title of a 2003 study by Nichols, et al., poses the question directly: "Is Online or In-class More Effective?" (378). The Nichols study, which focused on the use of a web-based tutorial for the delivery of information literacy instruction for the State University of New York at Oswego, found that there really was no difference between the two modes of instruction as measured by pre--and post-test results (385). In other words, the students in the tutorial group learned as much as the students who sat in the classroom. Another 2003 comparison of online versus classroom delivery also found "no observable difference in ... test performance" (van Schraik et al. 11). The van Schraik study also found no difference in the attrition rate between online and classroom environments (11).

Several studies published prior to 2003 tell of much the same experience in other instructional contexts. A 2000 study by Leasure, et al., found "no significant difference in examination scores" between students taking a nursing research course on the web and those taking it in class (n. pag.). This conclusion was based on the results of three multiple-choice examinations as well as final course grades (n. pag.). Clark and Jones, writing of their 2001 study of web and classroom versions of a public speaking course, also noted "no evidence that one format produced results superior to the other" (122). The results were based on speeches written and delivered by the students (111). Two studies published in 2002 provide more of a range of findings. In one study, learning outcomes of a teacher education course taught in both virtual and classroom formats were evaluated. The authors examined a conception of learning inventory together with a multiple choice test of critical reflection and, concluded there was no difference in terms of learning outcomes between the formats (Iran-Nejad and Xu 49). By contrast, in the other study, an evaluation of library skills learned either through an online tutorial or face-to-face instruction concluded that students in the face-to-face version scored consistently higher on pretests as well as post-tests (Churkovich and Oughtred 33-34).

The fact that there is no definitive winner is no cause for disappointment, however. To the contrary, knowing that both instructional formats are equally effective ought to be cause for encouragement to curriculum planners. In fact, the Churkovieh and Oughtred study endorses the view that "student learning appears to be highest when a course is neither all face-to-face nor all virtual but is instead a mixture of both" (33). This is encouraging to LSU Libraries, as the virtual LIS 1001 class is not a disembodied tutorial, but a course that is mediated by a librarian.

The LSU Experience

The impetus for promoting library skills and, specifically, information literacy at LSU has come from a multitude of directions. The Louisiana Board of Regents, invested by the 1974 Louisiana constitution (art. VIII, [section] 5) with the oversight of all degree programs in Louisiana's public institutions of higher education, has mandated that all students will have achieved "basic computer and informational literacy" upon graduation, leaving the precise method by which this is accomplished up to the various degree-granting institutions (Louisiana Board of Regents n. pag.). The Southern Association of Colleges and Schools, the agency charged with accrediting institutions of higher learning in the southeastern United States, requires that colleges and universities ensure "that users have access to regular and timely instruction in the use of the library and other learning/information resources" (Principles 9) and, further, that libraries provide assistance "consistent with the goal of helping students develop information literacy" (Criteria 55). Additionally, LSU, as stated in its own planning document, has committed itself to "promote among students, faculty and staff mastery of information technology and high skill levels in information retrieval, assessment, interpretation, and application" (LSU Planning Document n. pag.). With this level of institutional commitment as motivation, instructors and course designers within LSU Libraries are making every effort to meet the challenges laid out in the above mandates, and use of the web as an instructional platform seems the best way of immediately doing so.

During the period under study, eight different librarians taught the LIS 1001 course at LSU. However, the syllabus upon which each section is based is, in large measure, the same from section to section, thus allowing a reasonable level of comparison. In the present instance, that comparison is based on a study of the final examination grades earned over the study period. The exam grades of one hundred fifty-two students who took the course online during this period were compared with those of one thousand forty-five who took the course in a classroom. This is a roughly one-to-seven comparison. The reason for using the final exam grades and not the final course grades in this comparison is that the final exams for all sections are patterned after the same boilerplate examination, with each instructor adjusting for emphasis and other changes that may have been made to individualize the course. Besides, final course grades factor in assignments, exercises, and projects, the nature and weight of which vary from instructor to instructor. The average final exam grade for those students in the online version of LIS 1001 was 80.28, while the average for the classroom students was 84.22, not a statistically significant difference. Of greater significance was the difference in the distribution of grades. There were 10.51% more A's and 5.67% more B's on the final exam in the classroom course than in the online course. Conversely, the classroom course chalked up 7.25% fewer C's and 9.04% fewer D's than in the online course. The percentage of F's was virtually the same in each. In other words, it seems that, if final exam grades are any indication, students did better, at least in one respect, taking the course in the classroom than they did taking it online.

On the surface, these results would seem to vindicate Churkovich and Oughtred and support the conclusion that the classroom course is more effective than the online one. However, it should be noted that this is a preliminary and raw comparison. No attempt was made to control for instructor style, for example. It can easily be surmised, however, that at least one reason for the difference in final exam grades might be the dynamic that comes into play when student and teacher are face-to-face. And though this dynamic can be attempted in the visual environment, it cannot be the same as when facial expressions and vocal tones are real and in the present. Also, as noted above, though the syllabi and final examinations were materially similar, they were not exactly the same. Thus, the results of this rudimentary study can be considered only a the in a large evaluative mosaic and a prompt for further study.

Further Studies

Such a follow-up study is, in fact, being planned. The new study will control for instructor style by using the same instructor for both the online and classroom courses and will control for content and grading by using syllabi and assessments, which, after some adjustments to accommodate the practicalities of each environment, will be virtually identical. Remarks made by the students in their course evaluations give a hint of the carry-over effect of the course in both formats. The comments indicate a realization of the value of the skills taught in the course beyond the course itself. In this respect, the immediate impact of the course on at least some of the students seems to have been to create an appreciation of information in all its forms. Other comments, gathered through sporadic contacts with students of the classroom course years later, indicate a surprisingly enduring appreciation for the skills learned in the course. A formal study following up on and comparing the long-term after effects of classroom and web instruction is in order and would be helpful in determining not only whether one delivery format had more lasting impact than the other, hut also the depth of impact. It would be interesting to know, for example, whether the student had had occasion to transfer classroom knowledge to a situation unrelated to school.

Conclusion

Instructors who teach their classes online invariably report--and with some surprise--that their online courses take far more of their time than the traditional classroom approach. It is easy to see why. Gestures and other non-verbal signals must be recreated in exacting prose. The feeling of community, so easy to achieve in the classroom and so essential to the learning experience, is a painstaking matter in the online environment These come neither easily nor quickly to most people. Thus, to see online delivery as the solution to meeting increasing demand with limited resources seems a false hope. The only way online delivery solves the problem of limited resources is if the delivery method is totally automated. This is not how we teach at LSU. We may ultimately have to, but for now we do not want to sacrifice the invaluable interaction between students and teachers that often makes the difference between success and failure, regardless of the environment.

Students choose online instruction for its many advantages: the ability of students to "engage in more reflective thinking before having to answer or discuss issues" (Hiltz n. pag.), flexibility of scheduling, and convenience (Leasure et al. n. pag.). However, the primary reason to take any course should be to learn. The fact that, apparently in most cases, students are learning as much online as they are in the classroom must mean that those involved in online curriculum planning and design are not doing such a bad job.

References

Churkovich, Marion, and Christine Oughtred. "Can an Online Tutorial Pass the Test for Library Instruction? An Evaluation and Comparison of Library Skills Instruction Methods for First Year Students at Deakin University." AARL (2002): 25-38.

Clark, Ruth Anne, and David Jones. "A Comparison of Traditional and Online Formats in a Public Speaking Course." Communication Education 50 (2001): 109-124.

Hiltz, Starr Roxanne. "Impacts of College-level Courses via Asynchronous Learning Networks: Some Preliminary Results." For The Journal of Asynchronous Learning Networks. 1997. 15 Dec. 2003 <http://eies.njit.edu/~hiltz/>.

Iran-Nejad, Asghar, and Yuejin Xu. "A Comparison of Online and Regular Learning Contexts in a Course for Teacher Education Students." Technology and Teacher Education Annual. 1 (2002) 48-49.

Jorgensen, Daphne. "The Challenges and Benefits of Asynchronous Learning Networks." The Reference Librarian. 77 (2002): 3-17.

Leasure, A. Renee, Lisa Davis, and Susan L Thievon. "Comparison of Student Outcomes and Preferences in a Traditional vs. World Wide Web-Based Baccalaureate Nursing Research Course." Journal of Nursing Education. 39 (2000) 149-154. Wilson Web. H. W. Wilson. LSU Library 27 May 2003 <http://vnweb.hwwilsonweb.com/hww>.

Louisiana Board of Regents. "Attachment IV--Louisiana Board of Regents Statewide General Education Requirements." 26 April 2001. Louisiana Board of Regents. 18 Nov. 2002. <http://asa.regents.state.la.us/PP/Attachments/IV>. Louisiana Constitution. Art. VIII, [section] 5.

"LSU Planning Document," Release 2.0. October 2001. Office of Academic Affairs, Louisiana State University. 19 Dec. 2003 <http://www.lsu.edu/new/UPCDRFT.html>.

Nichols, James, Barbara Shaffer, and Karen Shockey. "Changing the Face of Instruction: Is Online or In-class More Effective?" College & Research Libraries 64 (2003) 378-388.

Southern Association of Colleges and Schools. "Criteria for Accreditation," 11th ed., Section 5.1.2. 2000. Commission on Colleges, Southern Association for Colleges and Schools. I March 2004 < http://www.sacscoc.org/SectV.htm>.

Southern Association of Colleges and Schools. "Principles of Accreditation: Foundations for Quality Enhancement," Section 3.8. January 2004. Commission on Colleges, Southern Association for Colleges and Schools. 1 March 2004 <http://www.sacseoc.org/pdf/principles%20of%20accreditation1.pdf>.

van Schaik, Paul, Philip Barker, and Scott Beckstrand. "A Comparison of On-Campus and Online Course Delivery Methods in Southern Nevada." Innovations in Education and Teaching International 40 (2003) 5-15.

Michael F. Russo, Louisiana State University

Russo, MLIS, is Instruction Coordinator for the LSU Libraries. His research interests include methods of spreading information literacy instruction on campus.