A comparative analysis of learning experience in a traditional vs. virtual classroom setting.

Abstract

This paper, based on an empirical analysis, compares students' attitudes and performance for a graduate Financial Management course, which was taught through two different modes (online and traditional) of course delivery. In an attempt to overcome the shortcomings of other studies comparing different modes of education, this study designed a similar learning environment for both online and traditional groups. Over three semesters a questionnaire, addressing four major criteria or indices (Web utility, interactivity, learning experience, and overall satisfaction) that captured nineteen attributes (variables), was distributed to both groups. After computing the indices and analyzing the collected data, the study concluded that there is no significant difference between the two groups of students' attitudes and performance.

Introduction

Instruction involves the development and communication of knowledge through a viable medium. The medium around which teaching/learning has transpired has undergone a variety of changes in form over the years, from home schooling to traditional classroom (face-to-face) to distance education and its new form, computer mediated communication (CMC), known as the "virtual" classroom.

In a traditional medium (classroom setting), instruction is accomplished through lecturing and interaction of students/students and students/faculty. However, recently with technological advancements in computing and the advent of the Internet, computers as a mere instructional tool are being transformed to a medium through which the instruction is being delivered. It is observed that the application of computers in instruction has resulted in a more active involvement of students in their own learning. Barr (1990), in support of this notion, in his discussion of educational reform, states that learning should be more "independent, individualized, interactive, interdisciplinary, and intuitive", and that computer-based classrooms support such learning.

The rise of online course offerings by colleges around the world has created both excitement and skepticism among students, educators, administrators, employers, and government officials for different reasons. Based on a survey 1 by the U.S. Department of Education's National Center for Education Statistics, the use of asynchronous Internet-based technologies in teaching has grown from 22 percent of institutions in 1995 to 60 percent of institutions in 1997-1998. The survey also points to a 33% rate of growth, within the same time period, among the higher educational institutions offering courses through distance education. A revolution in higher education is in the making.

Naturally, such dizzying growth in online course offerings breeds controversy. Among the hotly debated issues concerning the quality of learning are whether:

   1. The extent of learning that normally takes place in a traditional
   classroom could be replicated online; and

   2. Virtual classrooms allow the same amount of interactivity as their
   traditional counterparts.

Currently, online classes are taught in different formats. Some classes integrate a bulletin board and email on a Web page through which assignments are communicated to students. Students are required to use the email or bulletin board to explicate their completed assignments, pose their questions, and most importantly interact with one another. Other virtual courses offer online external links and assignments plus computer chat rooms on a real-time basis. This allows students to discuss group projects among each other, or get together with the instructor at designated times and discuss their questions and progress in the course. Some traditional classes also supplement their courses with Web-based resources. In either case, online capabilities have produced a pedagogical side benefit as a result of greater accessibility to the instructors and course contents, and the vast resources of the Internet.

Can technology as another medium supplant the traditional classroom? If so, could effective instruction be achieved through this medium? Regardless of the medium, effective instruction should clearly identify the learning objectives, impart knowledge, and provoke critical thinking. The current technological advancements have provided additional tools to educators for designing effective instruction. These tools (evolving on an almost daily basis) enable instructional designers and educators to produce a more appealing, consistent, and efficient learning environment, learning activities, and assessment tools for their students. If both traditional and technological mediums are able to attain effective instruction, then they could be transmutable. Effectiveness in education can be considered from many perspectives. It could be related to: (1) the "levels of achievements of educational goals" as articulated by Linke et al. (1984), or (2) the measurement of the "quantity" and "quality" of outcomes of learning in a given course as is advocated by Keegan (1990). In the literature, there is no consensus on what constitutes proper measures of effectiveness. However, it is generally agreed that such measures must be multidimensional. This study considers Keegan's notion of effectiveness and uses several indicators (Appendix A) to address and compare the effectiveness of both mediums.

Objectives of the Study

Keegan (1990) considers distance education as one of several modes of education with its own characteristics and discipline. Regardless of the mode, the goal of distance learning ought to be at least re-creation of the teaching-learning dynamic that takes place in a traditional classroom and in the process enhance, or keep unaltered the students' learning, performance, and interactivity. Using a CMC mode, the major objective of this study is to empirically test the extent to which this particular goal for the MBA Program's Financial Management course is accomplished. Considering the growth of distant learning in higher education in recent years, an understanding of this issue is useful for all parties (students, teachers, college administrators, businesses, and governmental entities) involved, since the CMC mode will undergo further scrutiny by those interest groups.

Hypotheses

The major issue in this study is an understanding of the impact of medium on student's learning, performance, and interactivity where medium is defined as CMC vs. traditional classroom. To investigate these questions, the following hypotheses will be tested:

   1. Is there a significant difference between the utility of the Web-based
   course contents for students attending virtual vs. traditional classrooms?

   2. Is there a significant difference in learning experience among students
   attending virtual vs. traditional classrooms?

   3. Is there a significant difference in interactivity among students
   attending virtual vs. traditional classrooms?

In other words, how does the medium of delivery (virtual/traditional classroom) impact interactivity? Interactivity is defined as connectivity the students feel with instructors and their peers. Sherry (1996) asserts "Without connectivity, distant learning degenerates into the old correspondence course model of independent study."

In regard to questions about the quality of online courses, the issue of interactivity is often a major concern as interactivity is considered to be an important pedagogical issue. Wagner (1997) states that instructors and program administrators seem to view interactivity as the defining attribute of CMC (p. 19). Critics of CMC usually point to lack of face-to-face interactions as a major shortcoming in distance education. Advocates of CMC, however, reject such assertions and provide evidence in support of effective interactivity in CMC. Wagner confirms a superior interactivity through CMC as opposed to a face-to-face classroom.

These three questions will be addressed through a comparative analysis of data over a three-semester period from a questionnaire (Appendix A), which was handed out to students in both control and experimental groups. Additionally, (1) to understand the students' general satisfaction with the course, an "overall satisfaction" index will be computed and compared, and (2) as a measure of performance in the course, the average final grades for both groups over the three semesters will likewise be computed and compared.

Prior Studies

By and large, the effectiveness of distance education courses have been studied in terms of the three following measures: Student outcome (such as test scores); Student attitudes towards learning; and Student satisfaction with the medium.

The bulk of research related to the effectiveness of online education points out that there are no significant differences between students' learning within different modes of knowledge delivery. Recent studies by Clarke (1999), Dobrin (1999), Dutton et al. (1999), Hoffman (1999), Navarro & Shoemaker (1999), Schulman & Sims (1999), Smeaton & Keogh (1999), and Wade (1999) that used different means of measuring learning effectiveness concluded that online students learned equally effectively as on-campus students. In a book published by Thomas Russell (1999) addressing such studies, the author concludes that distance learning is as effective as face-to-face education.

In a report sponsored by `the American Federation of Teachers and the National Education Institution and published by The Institute for Higher Education Policy (1999), the findings of such studies were questioned. The report indicated that, based on one or more of the following shortcomings, these studies' conclusions were flawed.

   1. Lack of control for extraneous variables;

   2. Subjects were not selected randomly;

   3. Instruments used in assessing student outcomes were invalid/unreliable;
and

   4. Lack of control for feelings and attitudes of students/faculty (reactive
effects)

In its approach, this study attempts to overcome these shortcomings through the design and control of the research environment.

The Course

Managerial Finance is a required MBA course at Frostburg State University. It is primarily attended by professional adult students with job and family responsibilities. The course requires the completion of two one-hour proctored examinations, a term project and one case. In addition, students are required to read selected articles and work on assigned problems on a weekly basis. For the term project and the case, students are allowed to work as teams of three to four students. The grade distribution for the course consists of twenty five percent each for (1) the two exams, (2) the term project, (3) the case, and (4) class participation (discussion of articles and related course concepts). Online students, thus far, have been from the pool of traditional students who for a variety of reasons (course scheduling conflict, saving travel time, moving out of area, and curiosity) decided to try this format of learning.

Research Design

To create a similar learning environment for both the control (traditional) and experimental (online) groups, similar requirements were instituted. All the materials for the Financial Management course were placed on two different Web sites. The course material for the online group was placed on the WebCt (commercially available courseware) platform. Additionally a separate Web site, containing the same course material as the WebCt site, was created from scratch on the University server. The course requires two proctored tests. Therefore, the online students were compelled to come to campus at specified dates to take those tests with the traditional students. The only difference between the two sites was that the WebCt site allowed asynchronous communication between the students, and the students/instructor, while this feature (online interaction) was not available on the site accessed by the traditional students.

For the traditional group most of the interaction was done face-to-face, while the instructional interaction for the online students was all through the asynchronous bulletin board and email. The two groups were required to interact with the course material over the Internet (read or download lecture material, assigned readings, solutions to homework problems, etc.). As a result, the factor of Novelty Effect (students working with something different: Internet) was eliminated (Merisotis & Phipps, 1999). Thus, except for the method of delivery of the subject matter, the two groups were exposed to identical course materials, instruction, and examinations.

Data and Research Methodology

Beginning in Spring 1999, a separate section of the Financial Management course was offered online for the first time. A series of questions (Appendix A) were developed to explain the following four major criteria:

   1. Learning environment [Web utility; (three questions, numbered 1 - 3)];

   2. Interactivity (three questions, numbered 4 - 6);

   3. Contribution of course materials and requirements to learning (nine
   questions, numbered 7 - 15); and

   4. Students' overall satisfaction with the course (four questions, numbered
16 - 19).

Appendix A was distributed to both groups (online and traditional) over the Spring, Summer, and Fall semesters of 1999. The class sizes for online sections were comparatively smaller as indicated in Exhibit I.

The results of the students' surveys were reported in 6 tables. Representative samples for online and traditional classes are shown in Tables 1 and 2. This data then was reorganized and compiled (Exhibit II) based on the four major criteria explained above. Exhibit II, shows the aggregate responses for the variables (questions) under each criterion. For example in Spring 1999, eight students took the Financial Management course online. The first and second criteria, "Web utility" and "interactivity" are each explained by three variables (questions). Therefore, there are 3 x 8 = 24 potential responses for each of those two criteria. See issue's website <http://rapidintellect.com/AEQweb/win01.htm>.

Using a Likert-Scale-based survey, each criterion will be quantitatively measured. In other words using all the variables under each criterion, an index, which represents students' opinions regarding that criterion, will be computed. If both groups generate similar indices for a given criterion, it could be inferred that the mode of course content delivery has no significant impact on that criterion. The Likert technique presents a set of attitude statements. Each student is asked to express agreement or disagreement on each question using the scales shown in Appendix A. For example if all online students respond to the three questions under the "Web utility" criterion by choosing the "strongly agree" scale, then 100% of the respondents strongly agree that the Web as a learning environment is a useful component of the course. Exhibit III, shows the compilation of such data for the three semesters and the four major criteria. The score for each criterion, as an index, represents an overall opinion of the students about how each mode of course delivery has effectively met the four major criteria. By analyzing and comparing the data in Exhibit III, the study will draw some conclusion regarding the previously stated three hypotheses.

Results

Exhibit III reports the computed indices representing students' opinions regarding each of the four major criteria. From this Exhibit, it can be shown that the majority of students (80% - 90%) in both groups and over the three semesters positively evaluated the course and its different attributes. Specifically, in reference to the first hypothesis, the results in Exhibit III clearly demonstrate that both groups (over the three semesters) believed that the Web-based information (Web as a learning environment) is a valuable component of the course. Based on these results, a case could be made in favor of providing Web-based materials for courses being taught in a traditional format.

The students' learning experience (subject of the second hypothesis) captured by the "course materials and requirements" criterion (index), as reported in Exhibit III, also indicates a majority in both groups (over the three semesters) thought that both mediums provided sufficient materials and requirements to enhance and contribute to their learning. With regard to the third hypothesis, in Exhibit III, again a majority of students in both groups (over the three semesters) agreed that both modes of course delivery allowed for effective interactivity among students and students/instructor.

Finally, the overall satisfaction criterion reflects the students' experience and impression of the course. This criterion, similar to other criteria, was computed and tabulated in Exhibit III. Over the three semesters a majority of students in both groups indicated that they were equally satisfied with the vigor and usefulness of the course.

Furthermore, this study compared the final grades of both groups (online and traditional) for each semester. The average final grade for both groups in the Spring and Summer semesters was B. However, in the Fall semester the online students' average final grade was A, while that of the traditional students was B. Based on this information, this study confirms the findings of other studies such as Schulman & Sims (1999), and Smeaton & Keogh (1999) who used grades as measures of performance, and found that there were no significant differences between the performance of the two groups who took their courses in different modes of course delivery. Therefore, online courses have the potential of providing comparable learning experiences for students regardless of the mode of course delivery. See issue's website <http://rapidintellect.com/AEQweb/win01.htm>.

Conclusions

This study collected data on four major criteria (Web utility, interactivity, course material and requirements, and overall satisfaction) representing nineteen attributes (variables) that addressed different concerns of two groups of students (online and traditional) taking a graduate Financial Management course over three semesters. Using the proposed research methodology, this study calculated four indices as measures of the four major criteria. These indices were compared in a pair-wise fashion for the two groups and for each semester. From this comparison it was concluded that there were no significant differences between the two groups' opinions regarding their feelings about the Web utility, interactivity (students/students, and students/instructor), learning experience, and overall satisfaction for the Financial Management course delivered on-site or online.

Footnotes

(1.) U.S. Department of Education, National Center for Education Statistics, Distance Education at Postsecondary Education Institutions: 1997-1998, December 1999.

(2.) Such as "Novelty Effect" which, refers to increased participation, interest or motivation of students working with something new, and "john Henry Effect" which, refers to either students or teachers in the control group performing beyond normal expectations, as they are being challenged to compete with something new.

References

Barr, D. (1990), A Solution in Search of a Problem: The Role of Technology in Educational Reform, Journal for Education of the Gifted, 14(1), 79-95.

Clarke, D., Getting Results with Distance Education, The American Journal of Distance Education, Vol. 12, No. 1, 1999.

Dobrin, J., Who's Teaching Online, ITPE News - Vol. 2, Issue 12, June 22, 1999.

Dutton, J., Dutton, M., & Perry, J., Do Online Students Perform As Well As Traditional Students?, submitted for publication, North Carolina State University, 1999.

Hoffman, K. M., What Are Faculty Saying? eCollege.com - May 1999.

Keegan, D., Foundations of Distance Education. New York: Routledge, 1990.

Linke, R., et al. (1984), Report of a Study Group on the Measurement of Quality and Efficiency in Australian Higher Education. Canberra: CTEC, p. 19.

Merisotis, J. P., & Phipps, R. A., What's The Difference? Outcomes of Distance vs. Traditional Classroom-Based Learning. The Institute of Higher Education Policy, April 1999.

Navarro, P. & Shoemaker, J., The Power of Cyber Learning: An Empirical Test, Journal of Computing in Higher Education, 1999.

Russell, Thomas L.,. The No Significant Difference phenomenon. Chapel Hill, NC: Office of Instructional Telecommunications, North Carolina State University, 1999.

Schulman, A. H. & Sims, R. L., T.H.E. Journal, Vol. 26, No. 11 June 1999.

Sherry, L., (1996), Issues in Distance Learning, International Journal of Distance Education.

Smeaton, A. & Keogh, G., An Analysis of the Use of Virtual Delivery of Undergraduate Lectures, Computers and Education, Vol. 32, 1999.

U.S. Department of Education, National Center for Education Statistics, Distance Education at Postsecondary Education Institutions: 1997-1998, December 1999.

Wade, W., Assessment in Distance Learning: What Do Students Know and How Do We Know that They Know It?, T.H.E. Journal, Vol. 27, No. 3, October 1999.

Wagner, E. D. (1997), Interactivity: From Agents to Outcomes, New Directions for Teaching and Learning, 71, 19-26.

Dr. Ashkeboussi, Associate Professor of Finance, has been teaching Corporate Finance online to graduate students for five years. He participated in the University System of Maryland Distance Education Fellows program, and in 1999 was named Distance Education Fellow by the Chancellor of the University System of Maryland.