On-line instruction: are the outcomes the same?

Institutions of higher education are offering more and more on-line courses to students. Do students receive the same quality of instruction with an on-line class as an on-campus class? Specifically, is there a difference in students' outcomes between a face-to-face class and on-line class? This study addresses that question by collecting and assessing data between students enrolled in the same course as it is delivered to one section face-to-face and another section on-line. The results of this study reveal that there are no significant differences in the students' outcomes between the two sections. These results support that the quality of on-line instruction is equal to face-to-face instruction. However, more research is needed to address issues related to on-line instruction.

**********

Technology has made a significant impact on how instruction is delivered in higher education (Teh, 1999). The majority of colleges and universities offer some form of web-based education (Palloff & Pratt, 2001). Each year, the number of courses being delivered over the internet is increasing. The teaching methodology of higher education faculty has changed as the result of teaching online (Greenwood, 2000). The important question to address in this movement of online course delivery is students' outcomes. Is the achievement level of students who take classes online equal to classes that are taught face-to-face? That is the question addressed in this study.

Online instruction is housed under the auspice of distance education. Distance education began in the 19th century to provide students who could not attend traditional classes the chance to learn. The evolution of various instructional delivery methods of distance education has included such mediums as correspondence courses, radio, television, and videotapes (Fender, 1999). Then in the 1990s, many higher education distance education programs adopted internet-based delivery as the new mode to deliver courses (Khan, 1997; Moore & Kearsley, 1996; Porter, 1997).

Today, the internet and its prevalence allow almost anyone to become a distance learner through online course offerings. This ease of use has been supplemented by the development of course management software with the purpose of making online courses easy to development and manage from the instructors' perspective and easy to undertake from the students' perspective.

There are several Computer Management System (CMS) program being used for online classes with Blackboard being the most popular (Olsen, 2001; Young, 2002). These CMS such as Blackboard is a structured format that allows instructors to post announcements, assignments, course documents, faculty credentials, and lecture notes that can be easily accessed by students. Blackboard also is structured so small groups can be formed and group work can be an integral part of the how the course is taught. It allows the opportunity for direct communication between instructor and students and between students through e-mail, discussion boards, and the virtual chat room.

So all the variables were in place (availability, convenience, ease of implementing, etc) to take distance learning to levels it has never experienced in its path. There has been an accelerate rate of course offerings through online delivery and it appears to be growing exponentially (Bennett, 2001; Boettcher, 1996; Hiltz; 1997; Lewis, Snow, Farris, Levin, & Greene, 1999; National Center for Education Statistics, 2002).

Interestingly, with the increasing acceptance of online classes (Owston,1997; Rossman, 1992) few studies have examined the effectiveness of online courses and whether the achievement levels of meeting the courses competencies and objectives were met (Navarro & Shoemaker, 1999; Schulman & Sims, 1999; Sener & Stover, 2000; Smeaton & Keogh, 1999; Russell, 1999; Spooner, Jordan, Algozzine, and Spooner, 1999). However, little research exist examining issues related to student satisfaction (Kirtley, 2002). This study was conducted by comparing the outcomes of students when one section of a course was taught face-to-face and one section of the same course was taught online.

Method

Subjects

The participants were a total of 52 students who were evenly divided into two sections of the same graduate course that dealt with the topics of teacher leadership and communication. The students were randomly assigned to the each of the two sections. One section was delivered the traditional face-to-face and the other section was delivered totally online. The demographics of the students in both sections were very close with the face-to-face section being 80% female with an average age of 34 and an average of nine years of teaching experience. The online section was 75% female with an average age of 33 and average of eight years of teaching experience.

Design and Procedure

Students were assigned various course requirements that included: designing and publishing a personal professional webpage, writing a professional mission statement that included clearly defined goals, conducting interviews, producing a powerpoint presentation, writing a research paper, article critiques, midterm, and final examinations. These requirements and products were organized in a portfolio and collected by the instructor. Three outside evaluations assessed these portfolios for their overall quality by rating them on a scale of one to seven with a one being the low and a seven being a high.

Students were pre-assessed at the beginning of the course and post-assessed at the end of the course in a self evaluation that examined their level of expertise in the course's competencies and objectives. This self-evaluation was conducted on a likert scale (1=no mastery - 7=total mastery). These evaluations were conducted on the first class meeting and the last class meeting after the students had completed their final examination.

Data for course evaluations were collected and shared with the instructor. Course evaluations are conducted on all course offerings at the university where this study was conducted. Students responded to questions that addressed their satisfaction with the different aspects of the course delivery. A liken scale of 1-7 was used (1=strongly disagree-7strongly agree). Questions such as instructor's preparation, availability, amount of work, etc., with the final question being the instructor's overall effectiveness.

Results

The data generated from this study indicated that there was no significant difference between the face-to-face section and the on-line section. The outside evaluators overall assessment of the face-to-face portfolios was an average of 6.00 (with a low of a three and a high of a seven) and for the on-line section the overall assessment was an average of 6.25 (with a low of a three and a high of a seven).

The self assessment data for the pre-assessment was an overall of 2.9 for the face-to-face section and 3.0 for the on-line section. For the post assessment, the face-to-face section overall score was 6.6 and for the on-line an overall score of 6.7 was calculated. These results once again reveal no significant difference between the two sections.

The distribution of final grades for the face-to-face section was as follows: 19 students earned an A; 6 students earned a B; and 1 student earned a C, For the on-line section 20 students earned an A and 6 students earned a B. Once again, there is no significant difference in the grade distribution between the two sections.

Results of the course evaluations administered by the university reveal no significant difference in students' satisfaction between the two sections. The score for the instructors overall effectiveness for the face-to-face section was 6.7 and for the on-line section the score was 6.8. The other areas of students' satisfaction addressed by the instrument revealed very similar results with none being at the significant level.

Discussion

The current trend in higher education is to provide more on-line course offerings. Institutions of higher learning are responding to student demand and are offering more and more courses on-line and are even increasing the number of complete degree programs being offered on-line. However, as on-line course offerings continue to increase, there needs to be research that examines the quality of the course offerings and the student outcomes. This research studies addresses those questions and the results support that students' outcomes are equal between face-to-face sections and on-line sections of the same course. The results of this study are encouraging in supporting distance education as providing educational opportunities for an ever-increasing diverse and mobile society. However, more research is needed to address the issues related to on-line course delivery and its impact on students and society.

References

Bennett, G. (2001). Student learning in the online environment: No significant difference? Quest, 53(1), 13.

Fender, D. L. (1999). Professional Safety. Distance Education, 44(10). Retrieved September 3, 2003 from EBSCOHost Academic Search Elite database.

Greenwood, T. M. (2000). E-Class: Creating a guide to online course development for distance education: An examination and analysis of change. Dissertations Abstract International, 61 (7A), 2576.

Hiltz, S. (1997). Impacts of college-level courses via Asynchromonous Learning Networks: Some preliminary results. Journal of Asynchronous Learning Networks [Online]. Available: http://eies.njit.edu/'hiltz/workingpapers/philly/ philly.htm

Khan, B. H. (1997). Web-based instruction. Englewood Cliffs, NJ: Educational Technology Publications.

Kirtley, K. E. (2002). A Study of Student Characteristics and Their Effects on Student Satisfaction with Online Courses Dissertation, West Virginia University, [On-line Abstract]. Available: http://etd.wvu.edu/templates/showETD. cfm?recnum=2538

Lewis, L., Snow, K., Farris, E, Levin, D., & Greene, B. (1999). Distance education at postsecondary education institutions 1997-98. National Center for Education Statistics [Online]. Available: http://nces.ed.gov/pubs2000/2000013.pdf

Moore, M. G., & Kearsley, G. (1996). Distance education: A systems view. Belmont, CA: Wadsworth.

National Center for Education Statistics (2002). A Profile of Participation in Distance Education: Postsecondary Education Descriptive Analysis Reports (NCES 2003-154). Washington, D.C.: U.S. Department of Education.

Navarro. P., & Shoemaker, J. (1999). The power of cyber learning: An empirical test. Journal of Computing in Higher Education, 11(1), 33.

Olsen, F. (2001). Getting ready for a new generation of course-management systems. Chronicle of Higher Education, 48(17), A25.

Owston, R. (1997). The World Wide Web: A technology to enhance teaching and learning. Educational Researcher. 26 (2), 2733.

Palloff, R. M., & Pratt, K. (2001). Lessons from cyberspace classroom: The realities of online teaching. San Francisco: Jossey-Bass.

Porter, L. R. (1997). Creating virtual classroom: Distance learning with the Internet. New York: Wiley.

Rossman, P. (1992). The emerging worldwide electronic university: Information age global higher education. Westport, CT: Greenwood Press.

Schulman, A. H., & Sims. R. L. (1999). Learning in an online format versus an in-class format: An experimental study. T H E Journal, 26(11), 54-56.

Sener, J., & Stover, M. (2000). Integrating ALN into an independent study distance education program: NVCC case studies. Journal of Asynchronous Learning Network 4(2).

Smeaton, A., & Keogh, G. (1999). An analysis of the use of virtual delivery of undergraduate lectures. Computers and Education. 32, 83-94.

Spooner, F., Jordan. L., Algozzine, B., & Spooner, M. (1999). Student ratings of instruction in distance learning and on-campus classes. Journal of Educational Research, 92(3), 132-140. Retrieved July 15, 2003 from EBSCOHost Academic Search Elite database.

Teh, G. (1999). Assessing student perceptions of Internet-based online learning environments. International Journal of Instructional Media, 26, 397-406.

Young, J. R. (2002). Pricing shifts by Blackboard and WebCT cost some colleges much more. Chronicle of Higher Education, 48(32), A35.

Louis L. Warren, Associate Professor, Associate Professor, Department of Curriculum & Instruction, College of Education. Harold L. Holloman, Jr., Assistant Professor, Department of Educational Leadership, East Carolina University.

Correspondence concerning this article should be addressed to Louis L. Warren, Associate Professor, Associate Professor, Department of Curriculum & Instruction, College of Education, East Carolina University.