Self selection of learning mode in an online course.

Abstract

Online learning approaches differ depending on whether the audience is a higher education or corporate one. In higher education, the favored approach is a collaborative, cohort model, intended to build community between learners. Distance Learning Assessment and Evaluation is one of the required courses that learners must complete in the DLDD program offered by the University of Washington. In this study, the investigator was interested in the effects of self-selection of self-paced versus a collaborative-group learning mode on class performance. Forty-one post-graduate professionals participated in this course. Initially, 71% of the learners chose the self-paced option, and 29% chose the collaborative-group option. Performance comparisons show no significant differences between the groups, apart from a slight bias toward the self-paced mode. These data refute the popular conception that the most effective online learning model for higher education is the "community of learners" model.

Introduction

Professionals outside of the academic community have recognized that there are distinct differences between online learning intended for corporate training and online learning intended for a higher education audience (Johnson et al., 1998). Corporate training favors self-paced methodologies, in which learners interact primarily with computer-based simulations and exercises, with infrequent or no interaction with other people (American Society for Training and Development, 1999, 2000; Schreiber & Berge, 1998). Self-paced methodologies allow learners to complete learning activities without the confines of start and stop dates, at the pace the learner determines, and without the need to spend time organizing collaboration with others (American Society for Training and Development, 1999, 2000; Johnson & Ruskin, 1977).

In contrast, the literature on best practices in online learning for higher education audiences suggests that courses should be (and, in reality, usually are) cohort-based (American Federation of Teachers, 2000; Gunawardena, 1992; Institute for Higher Education Policy, 2000; McCormack & Jones, 1997; Moore, Winograd, & Lange, 2001; Palloff & Pratt, 1999, 2001; White & Weight, 2000). Such courses have defined start and stop dates and learners are expected to complete assignments at the pace set by the online instructor. Furthermore, many higher-education online learning offerings also require collaboration between learners, such that activities often are completed in groups (Moore, Winograd, & Lange, 2001; Palloff & Pratt, 1999, 2000; White & Weight, 2000). Unlike the multimedia-rich exercises typically found in self-paced, corporate training, activities most often found in cohort-collaborative higher education models include discussion forums, text-based presentations, and web links. Interaction in this model usually refers to text-based discussion, synchronous chat, file transfer, and email between learners and with the online instructor. Engagement, therefore, largely depends on the ability and willingness of learners to interact with each other.

The rationale behind the cohort-collaborative model is the notion that working collaboratively helps build a "community of learners" (American Federation of Teachers, 2000; Gunawardena, 1992; Institute for Higher Education Policy, 2000; McCormack & Jones, 1997; Moore, Winograd, & Lange, 2001; Palloff & Pratt, 1999, 2001; White & Weight, 2000). However, the imposition of due dates and forced collaboration may build unnecessary boundaries, limiting the "anytime" in the popular catch phrase "anytime, anyplace" that is often associated with online learning.

The cohort model may be favored by institutes of higher education because of reasons such as budget, faculty resources, and, perhaps most significantly, the similarity with traditional, cohort-based face-to-face classes. It may be that the community of learners model is more beneficial to the online instructor or administrators than to the learners themselves. A simple way to determine if learners prefer to work in a self-paced fashion or in a group is to offer an online course in both modes and let the learners choose the mode in which they prefer to complete the course.

The University offers a certificate program in Distance Learning Design and Development (http://www.extension.washington.edu/extinfo/certprog/dld/dld_main.asp). Distance Learning Assessment and Evaluation (DLDD 103) is one of the required courses that learners must complete in order to earn their certificates. Before starting the course, learners take a pretest. The purpose of the pretest is two-fold: (1) It allows learners to self-select whether they want to take a self-paced or collaborative-group version of the course, and (2) it provides the online instructor with information regarding the incoming knowledge repertoires of the learners.

Such an arrangement allows for the comparison in performance between learners who choose to complete identical coursework in a self-paced mode and those who choose to complete coursework in a collaborative-group mode. Although some studies pertaining to online learning rely on self-report/survey data (for example, LaMonica, 2001), the focus of this study was the comparison of actual performance on the course activities. Because the DLDD program objectives are very practical-learners are expected to design and develop a distance-learning project in order to complete the program and thereby earn the certificate-and because the course activities are designed to teach practical, hands-on skills, the investigator was more interested in what learners could actually do (procedural knowledge) than what they say they can do (declarative knowledge). The purpose of this study, therefore, was to determine if a self-selection of collaborative-group/ community-of-learners model resulted in better learner performance outcomes when compared to the performance of learners experiencing the same content in a self-paced model.

Method

Subjects

Forty-one learners participated in this study. The learners were a mix of corporate trainers (6), professors (18), high school teachers (10), and representatives from nonprofit agencies (7). All learners held bachelor degrees, and some held advanced graduate degrees as well.

Materials

Materials used in this study included one required text and the online course content.

Procedure

Modes of Learning

The two modes of learning investigated in this study were the self-paced mode and the collaborative-group mode.

   * Self-paced mode. Learners self-selected this mode by checking a box next
   to this statement on the pretest: "Self-Paced Learner: I prefer to interact
   with course materials and assignments on my own at my own pace, perhaps
   with a little interaction with my instructor." In this mode, learners
   completed assignments on their own, at their own pace, and without
   collaborating with other learners taking the course. The only time-related
   requirement was that they were expected to complete all assignments prior
   to the course stop date. Unlike a traditional self-paced course, the course
   in this study had specified start and stop dates. Therefore, the self-paced
   mode used was a modification relative to how others have described
   self-paced learning (for example, Johnson & Ruskin, 1977).

   * Collaborative-group mode. Learners self-selected this mode by checking a
   box next to this statement on the pretest: "Collaborative Learner: I prefer
   to 'digest' course materials through interaction with my peers and
   instructor, and by completing both group and individual assignments." In
   the collaborative-group mode, learners completed some required course
   activities on their own, but completed the more intensive projects with
   others in groups. Assignment specifics are described in the next section.

Course Activities

Learners in the DLDD 103 course were provided with multiple opportunities to respond to what they were learning. Some of the course activities were not scored, whereas other assignments were scored. None of the assignments were mandatory, but learners needed to earn 92 out of a total of 131 points in order to successfully complete the course and continue in the program. How they chose to earn their points (that is, what activities they chose to complete) was completely at the learners' discretion. The scored activities that contribute to the 131 points included quizzes, discussion assignments and Web Quests.

   * Quizzes. The course included three scored multiple-choice/short-answer
   quizzes. All items on the quizzes pertained to the practical application of
   what was taught in each module of the course. Both self-paced and
   collaborative-group learners completed the quizzes on their own. Learners
   could earn up to a total of 50 points by accurately answering all of the
   items on these quizzes. Quiz scores reflect the number of items answered
   correctly.

   * Discussion Assignments. The course also included five discussion
   assignments. As with the quiz items, all discussion assignments centered on
   the application of knowledge to the learners' distance-learning projects.
   These assignments required the learners to respond to specific questions
   related to the topic of each module (some modules contained more than one
   discussion assignment). The self-paced learners completed these assignments
   in a "bulletin board" fashion-they simply posted their responses to the
   questions under discussion. In addition to posting their responses, the
   collaborative-group learners were prompted (but not required) to respond to
   other learners' posts, such that all posts and responses formed a
   discussion or conversation between the collaborative-group learners.
   Learners could earn up to a total of 51 points by adequately answering the
   discussion questions. Behavior-anchored rubrics were used to score these
   assignments to decrease subjectivity.

   * Web Quests. The most involved assignments in the course are Web Quests,
   which required more time commitment than the other two categories of
   activities. Web Quests required learners to visit multiple web sites,
   conduct research (as specified by the online course materials), and then
   synthesize and summarize their findings. In one instance, learners also
   create assessment items with online tools they have researched. Extensive
   guidelines and instructions are provided for all learners. Self-paced
   learners completed all aspects of these assignments on their own.
   Collaborative-group learners completed these assignments with others in
   their groups, following Jigsaw II methodology (Slavin, 1986). Each
   collaborative-group learner chose a single site to research, and answered
   only one part of the assignment. Once all members of the group had
   completed their independent research, they collaborated on their findings
   and submitted one group report. Learners could earn up to a total of 30
   points by accurately completing these Web Quests. As with the discussion
   assignments, behavior-anchored rubrics were used to score these
   assignments.

Course Grading

Because this course was designed to be performance- and application-based, course grades were determined exclusively by the number of points learners earned on the scored activities. No other factors, such as the number of assignments completed or participation, were used to calculate grades. Grades were awarded as follows:

   HD (High distinction): More than 95% or more than 124 points earned. A
   (Exceptional): 90 to 95% or 118 to 124 points earned. B (Satisfactory): 80
   to 89% or 105 to 117 points earned. C (Pass): 70 to 79% or 92 to 104 points
   earned.

Any learner who earned fewer than 92 points did not receive credit for the course and subsequently could not continue in the program.

Results

Several comparisons between the self-paced and collaborative-group results are reported here, including self-selection choice, average pretest-posttest quiz performance, average performance on course activities, overall performance in the course, and final grades.

   * Self-selection choice. When given the choice between self-paced mode or
   collaborative-group mode, 29 learners (71%) self-selecting the self-paced
   mode and 12 learners (29%) self-selecting the collaborative group mode.

   * Average pretest-posttest quiz performance. Although the pretest was an
   unscored activity, this comparison was investigated to determine if
   self-selection choice was correlated with entering knowledge repertoires.
   The comparison illustrated no difference between the groups. The self-paced
   learners answered, on average, 57% of the pretest items correctly, and
   scored 99% on the posttest. The collaborative group learners answered, on
   average, 58% of the pretest items correctly, and also scored 99% on the
   posttest.

   * Average performance on course activities. The average performance on
   course activities was virtually identical for the two groups. On average,
   self-paced learners scored 93% (47 points) on quizzes, 88% (45 points) on
   discussion assignments, and 91% (27 points) on Web Quests.
   Collaborative-group learners scored 95% (48 points) on quizzes, 83% (42
   points) on discussion assignments, and 100% (30 points) on Web Quests.

   * Overall performance in the course. When comparing overall performance, in
   terms of the number of points earned in the course, the results are similar
   for both groups. Self-paced learners averaged 88% (115) of the points
   available in the course, with a range from 70 to 98% (92 to 128 points).
   Collaborative-group learners also earned 88% of the available points, with
   a range of 71 to 98% (93 to 128 points).

   * Final grades. This comparison illustrated a difference. Of the self-paced
   learners, 31% earned HD, 31% earned A, 31% earned B, and 6% earned C. Of
   the collaborative-group learners, 33% earned HD, 17% earned A, 25% earned
   B, and 25% earned C.

Discussion

This informal comparison of the performance of self-paced versus collaborative-group learners suggests that neither model can be inferred as being more effective than the other. One comparison that suggests higher achievement through collaborative-group learning (Web Quest performance) actually reflects a bias. All collaborative-group learners completed these assignments, whereas 4 self-paced learners chose not to complete 1 or more of these activities. Therefore, the slight variation in performance is likely due to averaging based on noncompletion of these particular assignments by a few self-paced learners. Given that these assignments were more time consuming, it is likely that the self-paced learners who chose not to complete them did so because they had accumulated enough points through completing the other course activities.

While the difference in not significant, it is interesting to note that self-paced learners scored slightly higher on discussion assignments than did their collaborative-group counterparts. Discussions are the cornerstone of the community of learners model, yet the "bulletin board" reports received higher scores than did the truly learner-to-learner interactive discussions. This finding questions the utility of peer interaction on such activities, particularly since discussion assignment scores did not improve for the collaborative-group learners across modules. In contrast, the self-paced learners' scores on discussion assignments did improve across modules, as these learners clearly were implementing the online instructor's feedback in subsequent assignments. However, the sample size was too small to draw any definitive conclusions on this matter.

Another interesting finding was the comparison of final grades. When looking at the data for the self-paced learners, 93% of those learners scored 80% or higher (B or higher) in the course. When looking at the same data for the collaborative-group learners, only 75% of the learners scored 80% or higher. The groups are too small and of unequal size, thus statistical treatments of these data aren't possible. However, the suggestion that self-paced learning could result in higher overall course performance is in direct conflict with the community of learners model.

One of the benefits of completing coursework online is the ability to complete it anytime, anywhere. This benefit is only an option when coursework is completed in a self-paced manner. When coursework requires collaboration with other online learners or meeting intermittent, instructor-set deadlines, this flexibility is diminished. Some learners specifically chose the self-paced option for this level of flexibility. Anecdotally, it is worth noting that, before the end of the respective sections, 3 learners asked to be switched from collaborative-group mode to self-paced mode. No self-paced learners asked to be switched to the collaborative-group mode.

One cannot read too much into data based on a sample this small and with unequal groups. However, the study does demonstrate that the community of learners model of online learning requires more empirical study. Further investigation is required to examine the effects of mode of learning, particularly on longer courses of study.

References

American Federation of Teachers. (2000). Distance Education: Guidelines for Good Practice. Washington, DC: Higher Education Program and Policy Council.

American Society for Training and Development (1999). The ASTD Handbook of Training Design and Delivery. New York: McGraw-Hill.

American Society for Training and Development (2000). The 2000/2001 ASTD Distance Learning Yearbook. New York: McGraw-Hill.

Boettcher, J. & Conrad, R. (1999). Faculty Guide for Moving. Teaching and Learning to the Web. League for Innovation in the Community College.

Gunawardena, C. N. (1992). Changing faculty roles for audiographics and online teaching. American Journal of Distance Education, 6(3), 58-71.

Institute for Higher Education Policy (2000). Quality Online: Benchmarks for Success in Internet-based Distance Education. Washington, DC: IHEP.

Johnson, C., Hale, C., French, D., & Farr, G. (1998). Interact Based Learning: An Introduction and Framework for Higher Education and Business. Chicago, IL: Stylus.

Johnson, K. R. & Ruskin, R.N. (1977). Behavioral Education: An Evaluative Review. Washington, DC: American Psychological Association.

LaMonica, L. (2001). The Role of the Instructor in Web-based Instruction: Are We Practicing What We Preach? <http://www.geoeities.com/llamonica/instructorwbt.html>

McCormack, C. and Jones, D. (1997). Building a Web-Based Education System. New York: Wiley.

Moore, G.S., Winograd, K., & Lange, D. (2001). You Can Teach Online. New York: McGraw-Hill.

Palloff, R. M. & Pratt, K. (1999). Building Learning Communities in Cyberspace: Effective Strategies for the Online Classroom. San Francisco, CA: Jossey-Bass.

Palloff, R. M. & Pratt, K. (2001). Lessons From the Cyberspace Classroom. San Francisco, CA: Jossey-Bass.

Schreiber, D.A. & Berge, Z. L (1998). Distance Training: How Innovative Organizations Are Using Technology to Maximize Learning and Meet Business Objectives. San Fransisco: Jossey-Bass.

Slavin, R. (1986). Using Student Team Learning (3rd Ed.) Baltimore, MD: The Johns Hopkins University, Center for Research on Elementary and Middle School.

White, K.W. & Weight, B. H. (2000). The Online Teaching Guide: A Handbook of Attitudes, Strategies, and Techniques for the Virtual Classroom. Needham Heights, MA: Allyn & Bacon.

Dr. Ross is a faculty member in the Distance Learning Design and Development (DLDD) program. In addition, she serves as adjunct faculty for Southern Illinois University and Seattle Pacific University.