Asynchronous learning networks and student outcomes: the utility of online learning components in hybrid courses.

The current research focuses on the impact that learning management systems (LMS), specifically the Blackboard interface, are having on courses in psychology. Blackboard provides instructors with access to a powerful web-based instructional platform. One of the main benefits to students is the unfettered access to virtually anything an instructor presents in the classroom. For example, access to syllabi, course notes, interactive demonstrations, handouts, audio or videotaped lectures are all possible via this interface. Currently, few empirical studies have examined the impact of LMS on objective measures of student learning. The current project examines the relationship between the frequency of usage of these various utilities and student performance in a hybrid class. Results revealed a significant positive partial correlation between overall usage and their exam scores. The implications of these findings are discussed with respect to the current course; however, a discussion of the broader pedagogical implications is included as well.

**********

By now it is obvious to most in higher education that technology is transforming education delivery in profound ways. Duderstadt (1999) correctly asserts that the most significant technological development is the way that the former constraint of time and space has been removed by networking capabilities. Though some faculty remain skeptical about these changes, few can deny that these technologies are transforming the way learners learn and the way teachers teach. Recent statistics clearly indicate that access to various online educational opportunities is increasing. A study by the National Center for Education Statistics surveyed 4,130 two and four-year degree-granting institutions and found that 56% (2,320 institutions) offered distance education courses of some type during the 2000-2001 academic year. An additional 12% (490 institutions) indicated plans to offer distance education courses at some point in the next three years. It should be noted that these percentages reflect numerous varieties of distance education and not just online technologies, however, the study did reveal that of the 68% of the institutions are either currently offering or planning to offer distance education in the next three years, 88% plan to increase or start offering courses using asynchronous computer based instruction as the primary mode of delivery.

Furthermore, total enrollment in distance education courses went from 754,000 in 1995 to 1.6 million in 1998 (Harasim, 2000) and, according to the most recent data, over 3 million students enrolled in distance education courses in the 2000-2001 academic year (Waits & Lewis, 2003).

Because of the relative age of the field of online learning there is still much disagreement about the different types of online learning. In the past it has been lumped with other distributed learning modes such as correspondence courses and courses delivered via television etc. However, with the advances in technology, primarily increases in internet bandwidth, online technologies have increasingly become an integral piece of both distance education delivery and traditional, face-to-face courses. Interestingly, there is not one standard definition of what constitutes an online course. Most definitions or categorizations have to do with the ways online technologies are integrated into various courses. For example, Harasim (2000) identifies three modes of delivery; Adjunct Mode, Mixed Mode and Totally Online Mode. Adjunct Mode is described as traditional face-to-face courses that utilize online utilities to enhance course content. The difference between Adjunct mode and Mixed mode is the degree to which networking is integrated into the course. Whereas adjunct includes a few networking utilities that are added on as conveniences to the regular course, mixed mode courses include networking utilities as significant and well-integrated components of the overall course. The third mode includes those courses that are taught fully online and use computer networks as the primary environment for the course. Again, this is but one way of categorizing the different configurations of online technologies; other scholars do not make the distinction between the first two modes, instead labeling any course that blends online components with more traditional face-to-face instructional techniques "hybrid courses" (Swenson & Evans, 2003).

Regardless of whether a course is a hybrid or fully online there is much debate about the effects that these new technologies are having on traditional face-to-face courses. Some argue that we are in the midst of a true paradigmatic shift regarding the nature of learning in general and that, regardless of the role technology plays, the networked collaborative learning model has changed the way learners learn and along with it the way teachers teach. Despite the popularity of these technologies, relatively little research has examined their relative influence on objective measures of student learning. Much of the existing research has focused only on students' self-reported perceptions of learning (Hiltz, Coppola, Rotter, Turoff & Benbunan-Fich, 2000; Richardson & Swan, 2003; Wu & Hiltz, 2004). Those that have used objective performance measures have found somewhat dubious results for the courses that employ online pedagogies. For example, a number of studies have examined the relationship between student participation in online courses and grades and found no significant relationship between the two (Davies & Graft, 2005; Picciano, 2002). Many others have compared online and/or blended classes to traditional classes and found mixed results. For a comprehensive review of this research see Fjermested, Hiltz and Zhang (2005).

Though the focus of the research was mainly exploratory in nature, the current research sought to answer one obvious question: Does students' use of the components of a Blackboard platform relate positively to an increase in exam scores. To test this, students' frequency of Blackboard usage over the course of a ten week term was correlated with their overall grade in a general psychology class while controlling for their overall GPA.

Methods

Participants

The sample consisted of 80 students (37 males and 43 females) enrolled in two introductory psychology courses at a medium-sized university in the Northwest. Students included 35 freshman, 33 sophomores, 3 juniors and 8 seniors.

Procedure

The current research focused on two introductory psychology courses that were designed as traditional, face-to-face courses but included various online elements available to students through the Blackboard interface. Components available to students included the contact information of the instructor, email access to the instructor and other students, various course information regarding office location and the office hours of the instructor. In addition, all documents presented in class were also available online. For example students could view all class outlines, handouts, group and individual assignments and study guides. Students also received reminders about assignments and exams via the email and announcements functions within Blackboard. Though the Blackboard site was well-integrated into the course and students were encouraged throughout the term to utilize the site, they were not required to use it to pass the course.

Student usage of the Blackboard site was assessed using the tracking records function that is available to instructors within the Blackboard interface. This utility provides instructors with detailed information on the number of times (hits) students access the Blackboard site as well as the specific areas they access. Individual performance data was also collected for each student. Individual scores on three exams were also collected and averaged to create an overall exam performance score. In addition, each student's grade point average (GPA) was also obtained from students' transcripts.

Results

Blackboard Usage

Total Blackboard usage ranged from 0 to 508 with an average of 88.29 hits and a standard deviation of 76.42. Females used Blackboard significantly more (M=106.23, SD=89.17) than did males (M=67.43, SD=52.03) t(78)=-2.33, p<.05. There were no significant differences between school year and Blackboard usage F(3,75)=1.20, p=n.s.

Student Performance

Average grade in the class was a 79% (SD=.11). Students average GPA was a 2.88 (SD=.52). Final grades for the course were significantly higher for females (M=.82, SD=.09) than males (M=.76, SD=. 12) t(78)=2.77, p<.01).

Blackboard Usage and Student Performance

To test the main hypothesis, a partial correlation using students' overall GPA as a control variable revealed a small but significant correlation (r=-.23, [r.sup.2]=.05) between students' total amount of Blackboard usage and their final grade in the course. Furthermore, while controlling for GPA the correlation between Blackboard usage and the final grade for the course was stronger for men r=-.28 than for women r=.12.

Discussion

Although the current study provides statistical evidence for the benefits of the Blackboard interface, the overall effect size is small. Consequently, the results should be interpreted cautiously. Also, given the dynamic nature of individual classes, the generalizability of these findings is dubious. However, using exam performance as the measure of success, these findings indicate that students who used Blackboard performed better on exams than those who used Blackboard less frequently. Though the effect size was small this is one of the first empirical studies to establish a link between students' usage of online components and their overall success in the course. This is significant because few of the existing studies on LMS have utilized objective measures of student learning. Furthermore, it appears that Blackboard usage is more beneficial to males than it is for females though this finding should be viewed cautiously as females tended to use Blackboard more frequently and scored higher overall than did males. It's possible that females utilize Blackboard differently than do males, though the limitations of the tracking functions within the Blackboard platform limit the ability to explicate these differences. These findings suggest that individual differences in learning styles may influence both how students utilize online components as well as the degree to which students derive benefit from them. Future studies could examine the relationship between learning styles, usage of online components and student success.

Again, exam performance is but one method of assessing student learning so it is also possible that the short-term gains in student learning are negligibly influenced by Blackboard usage and that the real benefit shows up in students' long term retention of course material. Future research should include post-class follow-up measures to assess student retention of material. Another drawback of the current research is that although Blackboard provides data on the amount of usage, there is no way to measure the quality of that usage. That is, to be counted a student need only click on the contents tab, there is no way to know the difference between a student who opens a folder within Blackboard simply to see what's new and a student who spends three hours studying that day's class notes. Future research could address this by asking students to self-report not only on how often they use Blackboard but in what ways they utilized the contents of the Blackboard site. In addition, although the Blackboard site is readily available on campus or any system connected to the internet, some students may not have had as many opportunities to access the internet or had internet connections that did not allow convenient access to the materials. Furthermore, the current research focused only on Blackboard as a convenient, but not necessary element of a traditional classroom. Future research could assess the effectiveness of web-based courseware in classes that more fully integrate online technologies.

Despite these drawbacks, the findings are interesting in that they provide evidence for the utility of learning management systems. This is significant because few empirical studies to date have documented the effect of these systems on objective measures of student outcomes. This study, though limited in its generalizability, provides a foundation for future research on these increasingly popular pedagogical tools.

Correspondence concerning this article should be addressed to Dr. Daniel L. DeNeui at [e-mail].

References

Davies, J. & Graff, M. (2005). Performance in e-learning: Online participation and student grades. British Journal of Educational Technology, 36(4) 657-663.

Fjermestad, J., Hiltz, S. R. & Y. Zhang (2005). Effectiveness for students: comparisons of "in-seat" and ALN courses. In: Hiltz, S. R. and R. Goldman (Eds.), Learning Together Online: Research on Asynchronous Learning Networks, 39-80. Mahwah, N J: Erlbaum, 2005.

Harasim, L. (2000). Shift happens: Online education as a new paradigm in learning. Internet and Higher Education, 3, 41-61.

Hiltz, S. R., Coppola, N., Rotter, N., Turoff, M. & Benbunan-Fich, R. (2000). Measuring the importance of collaborative learning for the effectiveness of ALN: A multi-measure, multi-method mpproach. Journal of Asynchronous Learning Networks, 4(2). Retrieved on March 30, 3006, from http://www.sloan-.org/publications/jaln/ v4n2/v4n2_hiltz.asp.

Picciano, A. (2002). Beyond student perceptions: Issues of interaction, presence, and performance in an online course. Journal of Asynchronous Learning Networks, 6(1). Retrieved on March 30, 2006, from http://www.sloan-c.org/ publications/jaln/v6n1/v6nl_picciano.asp

Richardson, J. & Swan, K. (2003). Examining social presence in online courses in relation to students' perceived learning and satisfaction. Journal of Asynchronous Learning Networks, 7(1) Retrieved on March 30, 2006 from http/www.sloan-c.org/publications/jaln/v7n1/v7nl_richardson.asp

Swenson, P.W. & Evans, M. (2003). Hybrid Courses as Learning Communities. In Sorel Reisman (Ed.), Electronic Learning Communities Issues and Practices (pp. 27-72). Information Age Publishing.

Waits T. & Lewis. L (2000-2001) U.S. Department of Education, National Center for Education Statistics. Distance Education at Degree Granting Postsecondary Institutions: 2000-2001, NCES 2003-017. Retrieved March 30, 2006 from http://nces.ed.gov/pubs2003/2003017.pdf

Wu, D. and Hiltz, S.R.(2004). Predicting learning from asynchronous online discussions. Journal of Asynchronous Learning Networks, 8(2) 139-152 Retrieved on March 30, 2006, from http://www.sloan-.org/publications/jaln/v8n2/ v8n2_wu.asp

Daniel L. DeNeui, Ph.D., Southern Oregon University, Ashland, Oregon. Tiffany L. Dodge, B.S., Oregon Health & Science University, School of Nursing, Ashland, Oregon.