An empirical investigation of student achievement and satisfaction in different learning environments.

The purpose of this study was to investigate the effects of three different methods of instructional delivery (online instruction, traditional face-to-face instruction, and a combination of online and traditional instruction) on student achievement and satisfaction levels used in an undergraduate wellness course at a Midwestern university. Differences in the student ratings of the course and instructor, quality of learning, quality of communication, and support were also examined. One hundred fifty-three undergraduate students (71 men, 82 women; between the ages of 18 and 55 years, M=22.5 years, SD=7.0) completed a survey for this study. A survey was developed to examine student demographics, student perceptions of online learning, and student satisfaction levels. Comparing the mean scores of a written pretest and post-test among three groups was used to determine the content knowledge achievement of students. A one-way analysis of variance (ANOVA) and Post hoc Scheffe multiple comparisons were conducted to compare the effects of the three different methods of instructional delivery on student achievement and satisfaction levels. The results of this study indicated that students in the online learning group and the combined learning group had statistically significant higher levels of achievement than students in the traditional learning group (p<.01). Students in the combined learning group had significant greater satisfaction levels with their overall learning experience than students in the traditional learning group (p<.05). But, no significant differences were found between the online learning and traditional learning groups. Most students indicated that they would like to see an online option when enrolling for the course in the future. These findings suggest that a well-designed online course and a web-enhanced residential course can be effective in teaching wellness.

**********

Over the past decade, advances in the Internet and information technologies have significantly facilitated student learning and teaching in colleges and universities throughout the world. Today, access to the Internet and information technologies is widely available in homes, schools, libraries and other settings accessible to students. Considering a large percentage of university populations working part-time or full-time and using technology on a more frequent basis than in the past, online education using the Internet and information technologies is becoming and increasingly popular tools for distance education to better meet students' needs, interests, learning styles and work schedules. Recent National Center for Education Statistics (NCES) reports demonstrate that online education availability, course offerings, and enrollments have been increased rapidly among institutions from K-12 to four-year universities since the 1990s (NECS, 2003).A comprehensive survey released by the Sloan Consortium indicated that online education would continue to grow at a rate of nearly 20%. In line with the national trend, more colleges and universities are increasingly adopting and implementing online education (Allen & Seaman, 2003).

Although a growing number of research studies have compared the effectiveness of online instruction to traditional face-to-face instruction, findings from these studies have been markedly mixed. A majority of the published studies show no difference in student performance and student satisfaction regardless of whether a course was taken traditionally or online, whereas others show advantage for online instruction or for traditional instruction. For instance, McFarland and Hamilton (2006) found no difference in student performance and satisfaction between students who studied online and students who studied in a traditional manner. Further, some studies found the online instruction group achieved better performance and higher levels of satisfaction than the traditional classroom (Zhang, 2005). However, some other researchers found negative effects of online education, including the fact that students in the online instruction were less satisfied and (Rivera & McAlister, 2001). The literature concerning online education is somewhat contradictory, with reports of both positive and negative learning outcomes. Furthermore, according to some researchers (Ungerleider & Burns, 2003) and the Institute for Higher Education Policy (IHEP, 1999), most of the studies are methodologically flawed (e.g., lack of experiment control, small sample sizes, lack of random assignment of students, poor designed dependant measures). Only a small percentage of the studies conducted on distance learning contain original, quantitative research (IHEP).

Online learning has become increasingly popular for instruction in traditional classes to enhance teaching and learning. Some studies found that integrating online components into traditional classes substantially improved communications, increased assess to Internet resources and provided a high level of student satisfaction. (Kaynama & Kesling, 2000; Reid & Woolf, 1996; Schrum & Lamb, 1996). Additionally, technology skills have become an important key to success in the modem workplace. Therefore, online education can provide students with an array of sources and increased opportunities to improve knowledge and skills in technology applications that are vital to the modern workplace (Gubbins, Clay, & Perkins, 1999; Johnson, Roach, & Homes, 1999).

Despite the proliferation of online learning in higher education, the little scientific, qualitative research has been conducted on student achievement, satisfaction and other attitude ratings of online learning in the areas of physical education and sport studies. Gaining knowledge of student perceptions of online learning and its effectiveness is essential in order to improve online teaching and student learning. Therefore, the purpose of this study was to investigate the effects of three different methods of instructional delivery (online instruction, traditional face-to-face instruction, and combination of online and traditional instruction) on student achievement and satisfaction levels used in the wellness course at a Midwestern university. Differences in student rating of the course and instructor, quality of learning, quality of communication, and support were also examined.

Methodology

Subjects

The participants in this study were 153 undergraduate students (71 men, 82 women; between the ages of 18 and 55 years, M=22.5 years, SD=7.0) who were required to take the wellness course as institutional graduation requirements at a Midwestern university. As the students registered into one of the three modes of instructional delivery based on their preference, they were divided into three treatment groups: online learning group, traditional learning group, and combined online and traditional learning group.

Instrumentation 

Following a review of related literature, we developed an Online Education Survey (OES) to obtain demographic makeup and background characteristics, student satisfaction, student perceptions of the instructional delivery, instructor's effectiveness, quality of communication, and support. Reliability was determined with a test-retest pilot study. To determine the test-retest reliability coefficient between two pilot surveys, a Pearson Product Moment Correlation was calculated. The test-retest reliability was r = 0.93. A Cronbach alpha coefficient value was also computed to determine the internal consistency of the two surveys. The Cronbach alpha coefficient value was 0.91. The reliability coefficients for this survey instrument were high. In addition to the OES, we modified and used the University's Student Evaluation on Teaching Survey to evaluate course contents, availability of the instructor, quality of learning experience, and grading process.

Procedures

Four sections of the wellness course were offered during the fall semester. One section received only online education with no face-to-face interaction between the instructor and students and among students. Another section was taught on campus with combination of online instruction and traditional face-to-face instruction. The rest two sections were taught on campus through a traditional face-to-face method. Nevertheless, all students in the three groups had the same instructor, requirements, learning objectives, and course materials such as exams, assignments and textbook.

Students in the online learning and combined learning groups were trained to use the WebCT and its features by the instructor and an instructional specialist during the first week of class. An online interactive, virtual tutorial program for the WebCT was also provided for the students. After training, the students in both groups were able to use the discussion board, e-mail, lessons, web links and other course materials. In the online learning environment, students were given each lesson with an interactive streaming online lecture, virtual lab, online quiz, online discussion and an extensive set of web links. Also, the immediate feedback from an online quiz was available for each lesson so that students can spend more time in areas where they needed to improve their understanding. An extensive set of Web links was also provided to help students explore research and locate information related to course content. All communication was stored and tracked for analysis after the research was concluded.

A pretest and post-test content knowledge test were given to all students in each treatment group to measure knowledge they were expected to master during the course. Comparing mean scores of a pretest and post-test among the three groups was used to determine the content knowledge achievement of students. In order to provide exam integrity, students were proctored. In addition, the survey questionnaire was administered to each student at the end of the semester to provide the student with enough time to become familiar with the course. We explained the nature of the survey to students and answered questions they had prior to administering the survey.

Statistical Analysis

Descriptive statistics (percentages, frequency distributions, means, ranges, and standard deviations) were utilized to analyze student demographic characteristics. A one-way analysis of variance (ANOVA) was conducted to compare the effects of the three different methods of instructional delivery on student achievement and satisfaction levels. When the results of the ANOVA test were statistically significant, Post hoc Scheffe multiple comparisons were conducted to determine where differences between means existed. Statistical significance was accepted at an alpha level of p<.05.

Results

Demographic Characteristics of Participants

One hundred fifty-three undergraduate students (71 men, 82 women; between the ages of 18 and 55 years, M=22.5 years, SD=7.0) completed the survey. Of the respondents, 31 (14 men, 17 women; M=30.3 years, SD=10.6) were from the online learning group, 82 (42 men, 40 women; M=20.4 years, SD=3.0) from the traditional learning group, and 40 (15 men, 25 women; M=20. 8 years, SD=5.0) from the combined learning group. The online learning group and combined learning group consisted of more female students than the traditional learning group. As shown in Table 1, we found the mean age of students in the online learning group was significantly higher than the other groups. Most students in all three learning groups were either freshmen or sophomores. The results of the pre-content knowledge test indicated that there were no statistically significant differences in the mean for test scores among the three groups, F(2, 150) = .2, p>.05. Using a one-way ANOVA, we found significant differences in Internet usage for educational tools among the three learning groups, F(2, 150) = 27.4, p < .001. Post hoc Scheffe multiple comparisons indicated that students in the online learning and combined learning groups used the Internet more often for educational tools prior to taking this course than students in the traditional learning group. Students in the online learning group had more experience in taking an online course prior to taking this course, F(2, 150) = 3.1, p <.05. Also, students in the online learning group had better technology skills prior to taking this course, F(2, 150) = 7, p < .05.

Student Achievement

All students in the three learning groups completed a pre- and post-course knowledge test to measure skills and knowledge they were expected to master during the course. The mean for the pretest score was 61.9%, while the mean for the posttest was 75.4%. As shown in Table 2, the mean difference between the pretest and posttest scores was 17.3 (SD = 8.3) for the online learning group, 11.4 (SD = 13.1) for the traditional learning group, and 17.8 (SD = 14.1) for the combined learning group. Using a one-way ANOVA, we found significant differences in the student achievement among the three groups, F(2, 150) = 5.6, p < .01. The results of the Post hoc Scheffe test show that students in the combined learning and online learning groups had a significant higher achievement than the traditional learning group, and no significant differences were found between the combined group and online group.

Student Satisfaction Levels

A five point Likert scale (5 = very satisfied, 4 = satisfied, 3 = neutral, 2 = dissatisfied, and 1 = very dissatisfied) was used to measure satisfaction levels of the participants with their overall learning experience including the overall quality of the instruction and the course. As shown in Table 3, students in the three learning groups provided positive ratings: the mean rating was 3.9 (SD = .7) for the online learning group, 3.7 (SD = .7) for the traditional learning group, and 4.2 (SD = .8) for the combined learning group (see Figure 2). Using a one-way ANOVA, we found significant differences in the means among the three groups F(2,150) = 4.8, p <.05. Post hoc Scheffe multiple comparisons indicated that students in the combined learning group showed significantly greater satisfaction levels than the traditional learning group, and no significant differences were found between the online learning and combined learning groups.

Student Evaluations of Learning Experiences

A five point Likert scale (5 = excellent, 4 = above average, 3 = average, 2 = below average, and 1 = poor) was used to measure students' ratings of their learning experiences. All three learning groups rated the quality of learning positively: the mean rating was 3.9 (SD = .8) for the online learning group, 3.3 (SD = .8) for the traditional learning group, and 3.7 (SD = .9) for the combined online and traditional learning group. The online learning group rated it significantly higher than the traditional learning group, F(2, 150) = 7.6, p<0.05 .All three learning groups rated the overall quality of the course positively: the mean rating was 4.2 (SD = .9) for the online learning group, 3.4 (SD = .8) for the traditional learning group, and 3.9 (SD = 1.0) for the combined learning group. We found that the online learning group and combined learning group rated it significantly higher than the traditional learning group, F(2,150) = 10.7, p<.05.

The researchers found a significant difference in the quality of communication with the instructor among the three groups. The online learning and combined groups rated it statistically significantly higher than the traditional learning group, F(2, 150) = 3.2, p<0.05. But, no significant difference was existed in the quality of communication with their peers. There were no significant differences in the instructor's encouragement and help among the three learning groups, F(2, 150) = 2.9, p>0.05; F(2, 150) = 1.6, p>0.05. Students in the online learning and combined learning group indicated that this course helped them improve their computer technology skills, F(2,150) = 62.1, p <.05.

Most students in the online learning group perceived that it was easy or very easy to navigate this online course and access its materials. Compared to the traditional class, 45% of students in online learning group reported that they put in more work or much more work, 36.4% said they put in equal work, and 19.4% indicated they put in less work. Also, most students in the online learning group indicated they were likely or very likely to take additional online courses. More than 90% of students in the online learning group indicated they were likely or very likely to recommend the online course to other students, based on their experience in this online course. All students in the online group were satisfied or very satisfied with accessibility of a computer and online library materials needed for this course. The majority of online students indicated the main reason to take this course was convenience and flexibility. Overall, most students in the online learning group are appeared to enjoy their online learning experience. One student stated that "I liked online components; it made me read the chapters carefully and not depend on the instructor to tell me the material." Regardless of the learning groups, most students indicated that they would like to see an online option when enrolling for the course in the future.

Conclusion

As online learning using the Internet and information technologies has been becoming increasingly popular for instruction in both distance education and the traditional class to enhance teaching and learning, it is becoming crucial to gain a better understanding of student perceptions of online learning and its effectiveness in order to improve online teaching and student learning. The purpose of this study was to investigate the effects of three different methods of instructional delivery (online instruction, traditional instruction, and combination of online and traditional instruction) on student achievement and satisfaction levels used in the wellness course at a mid-sized rural university. Findings indicated that there were significant differences in student achievement among the three learning groups. Students in the online learning and the combined learning groups had a significant higher achievement than students in the traditional learning group (p<.01). The finding of this study supports a number of the previously published studies because the online instruction group had a significantly higher achievement than the traditional classroom (Zhang, 2005).

Students in the combined learning group had significant greater satisfaction levels with their overall learning experience than students in the traditional learning group (p<.05). However, there were no significant differences found between the online learning and traditional learning groups. The finding of this study supports the previously published studies because there was no difference in student satisfaction between students who studied online and students who studied in a traditional manner (McFarland & Hamilton, 2006).

The findings of this study indicate that students in the online group rated statistically significantly higher on the overall quality of course, the quality of learning, and the quality of communication with the instructor than students the traditional learning group. The findings in this study suggest that a well-designed online course can be effective in teaching wellness. The findings of this study are somewhat limited by 'the lack of random assignment of students. Also, students self-selected a section of this course, and, therefore, may be predisposed toward more favorable attitudes toward their option of taking this course.

Despite the fact that online education is somewhat contradictory, with reports of both positive and negative learning outcomes, few would disagree with the rapid emergence and growing prevalence of this instructional medium within physical education and sport studies. Therefore, it is important to recognize the different advantages of each type of instructional delivery and emphasizes the use of online learning to supplement, enrich and enhance student learning. More importantly, it is essential to continuously investigate not only the effects of online learning on student achievement and satisfaction, but also factors influencing the achievement and satisfaction.

References

Aljadaani, H. (2000). A comparison of web-based and conventional-based training methods in a leading Midwestern company. Dissertation, University of Alabama.

Allen, E. & Seaman, J. (2003), Sizing the Opportunity: The quality and extent of online education in the United States, 2002 and 2003, Needham, MA: Sloan-C. Retrieved March 20, 2007, from http://www.sloan-c.org/resources/survey.asp.

Gubbins, J. Clay, M. & Perkins, J. (1999). The development of an online course to teach public administrators computer utilization. Online Journal of Distance Learning, 2(1). Retrieved February 10, 2007, http:www.westga.edu/-distance/gubbins22.html.

Institute for Higher Education Policy. (1999). What's the difference: A review of contemporary research on the effectiveness of distance learning in higher education, Washington, DC: Author.

Johnson, J., Roach, T., & Hames, R. (1999). Administering a keyboarding course on the would wide Web. Online Journal of Distance Education Administration, 1(1).

Kaynama, S. & Keesling, G. (2000). Development of a web-based Internet marketing course. Journal of Marketing Education, 22(2), 84-89.

McFarland, D. & Hamilton, D. (2006). Factors affecting student performance and satisfaction: Online versus traditional course delivery. Journal of Computer Information Systems, 46 (2), 25-32.

National Center for Education Statistics. (2003). Distance education at degree granting postsecondary institutions: 2000-2001, NCES 2003-017 (Washington, DC, US Department of Education, National Center for Education Statistics). Retrieved February 10, 2007, http://nces. ed.gov/pubs2003/2003017.pdf

Reid, J.E. & Woolf, P. (1997). Online curriculum development at shorter college: Areport from the field. Retrieved February 10, 2007, http://www. caso.com/iu/articles/reid02.html

Rivera, J.C. & McAlister, M.K. (2001). A comparison of student outcomes and satisfaction between traditional and Web based course offerings, in: Proceedings of the 2001 Information Resources Management Association International Conference, Toronto, Ontario, Canada, 770-772.

Schrum, L. & Lamb, T. (1996). Groupware for collaborative learning: A research perspective on processes, opportunities, and obstacles. Journal of Universal Computer Science, 2(10), 717-731.

Ungerleider, C., & Burns, T. (2003). A systematic review of the effectiveness and efficiency of networked ICT in education: A state of the art report to the Council of Ministers Canada and Industry Canada. Ottawa: Industry Canada.

Zhang D. (2005). Interactive multimedia-based e-learning: A study of effectiveness, American Journal of Distance Education; 19(3), 149-162.

Dr. Jon Lim, Minnesota State University, Mankato, MN. May Kim, University of Florida, Gainesville. Dr. Steve S. Chen, Morehead State University, Morehead, KY, Dr. Cynthia E. Ryder, Mayes University, Orange Beach, AL.

Correspondence concerning this article should be addressed to Dr. Jon Lim at [e-mail].

Table 1
Characteristics of Participants Prior to Taking the Course

Variable                  Online            Traditional

                      M [+ or -] SD        M [+ or -] SD
Age                   30.3 [+ or -]        20.4 [+ or -]
                          10.6 *                3.00
Content Knowledge   61.3 [+ or -] 11.2   62.1 [+ or -] 12.4

Variable                 Combined             F value         p value

                      M [+ or -] SD           (df = 2)
Age                 20.8 [+ or -] 5.0           34.7           0.001

Content Knowledge   62.1 [+ or -] 12.3          0.2            0.859

Note: M = mean; SD = Standard Deviation; an asterisk
(*) = significance using the Scheffe procedure.

Table 2
Differences Among Groups on Student Achievement

Groups         M      SD    Online   Traditional

Onlinte       17.3    8.3
Traditional   11.4   13.1     *
Combined      17.8   14.1     NS          *

Note: NS = nonsignificant differences between pairs of means, while
an asterisk (*) = significance using the Scheffe procedure.

Table 3
Differences Among Groups on Satisfaction Levels

Groups         M    SD   Online   Traditional

Online        3.9   .7
Traditional   3.7   .7     NS
Combined      4.2   .8     NS          *

Note: NS = nonsignificant differences between pairs of means,
while an asterisk (*) = significance using the Scheffe procedure.