Impacting distance learning success rates.

Abstract

Comparison of distance education (DE) and face-to-face (F2F) classes at a California community college found that a significant difference did exist between success and retention rates with the DE rates far lower than equivalent F2F classes. Further investigation discovered that the DE success rates for the school were in many cases significantly lower than the statewide DE averages. Changes in the delivery and accountability of the classes were instated for the fall 2003 semester resulting in a significant improvement in success rates.

Introduction

The College of the Sequoias (COS) developed an experimental distance education program to support the Language Arts department in 1996. Though absent of funding, training, mission or support, the DE program expanded to include the divisions of Math, Business, Child Development, Library, and the Government and Nutrition programs. It uses Blackboard[R] as its DE delivery software.

COS is a medium sized community (junior) college located in Tulare county in the southern end of California's San Joaquin valley. It has one of the geographically largest districts in the state and includes all of King's County as well. While centrally located, COS' demographics are nearly identical to all of the surrounding colleges which include the community colleges at Porterville, Bakersfield, Fresno City, Reedley, and West Hills. Statistical data from the Chancellor's office indicated the near identical demographics for each school for the spring 2003 semester for ethnicity, age group, and gender. [1]

Though typical in demographic to the colleges in its vicinity, COS far exceeds its neighbors in student success. In this research, success is defined as the student's completion of a course with an A, B, or C grade. In the spring 2003 semester, COS reported 22,629 successful student completions out of 32,291 attempts (the average student enrolls in 3 classes) for a success rate of 70.08% [2]. Though the success rates at COS were higher than its demographically identical neighbors, the success rates for COS' DE courses were significantly lower with 233 successes out of 500 attempts for a success rate of only 46.6%--a difference of 24.2 points. This significant difference between the success rates of the DE classes and the success rate of the school overall, along with national research indicating that DE success and F2F success should be the same (Beattie, Spooner, F., Jordan, Algozzine, & Spooner, M, 2002; see also Butler, 2004 and Scott, 2002), caused the administration at COS to consider deep cutbacks in the DE program and possible elimination. This consideration was the inspiration for this research. The two questions that this project sought to answer were 1) the true distinction of the differences in the success rates between the DE and the F2F classes, and 2) to identify ways of decreasing that difference without reducing the rigor of the DE classes.

Quantification of Success Rates

While distance education has been available for quite some time (Schrum, 2002), and has been well researched (Nather, 2003; Beattie, 2002; Scott, 2002), there has been a significant increase of late in the number of DE offerings and in the number of institutions now providing DE courses, programs, and entire degrees (Donahue, 2002). As such, over 3 million students (Waits, 2003) are currently involved in some form of distance education. The California Community Colleges have witnessed a 380% growth in the number of DE noncredit courses and a 180% increase in the number of DE students (152,690 in 2001-02) (Nather, 2003). Additionally, the landmark on-going research by Thomas Russell (n.d.) demonstrates a wide body of work indicating the equality of DE and F2F education. Yet, Nather (2003) did find that in the California community colleges, the average success rate for DE classes is 54% where the average success rates for F2F classes is 64%.

Appropriate comparisons of classes

The comparison method of choice for this research is to be as consistent as possible between class types, while maintaining the comparison within a division. In this research, the DE classes were compared with the same course offering when available. Because some DE classes are not offered F2F, and because a particular instructor may or may not also lead the same class in both venues or may lead different DE courses, there will be some variable in the statistical results. For this reason, the DE class statistics were combined and compared to the averages of all similar F2F classes and the results were totaled by division. F2F classes which are not offered as DE are excluded. All instructors are experienced in F2F class delivery, but only DE instructors are expected to have DE delivery training.

The research discovered a clear distinction of success rates between divisions within the college with some divisions achieving success above the state average. However, the Business Division DE success rates for the spring 2003 semester fell 20.7 points below the statewide DE average of 54; whereas the F2F success rates fell only 5.6 points below the statewide average of 64. Though low, the Business Division DE success rates were consistent. Recent DE success rates for the division are demonstrated below with the rate falling typically greater than 17 points below the state average: Spring 2003:-20.7 points behind statewide average; Fall 2002: -13.6; Spring 2002: -20.1; Fall 2001 : -17.8. Based on this information, the DE classes in the Business Division were isolated and examined.

Changes in presentation

The challenge of the Business Division was and continues to be to enhance the success rates of the DE offerings without diminishing the rigor or requirements of the same. Since the success rates of the DE classes are not dependant on the success rates of the F2F classes, no change was made to the F2F offerings.

The five business division DE offerings included three computer topics and applications classes led by two different instructors and two keyboarding classes each led by a single instructor. All three instructors are certified in distance learning instruction from Cerro Coso Community College's CC Online program. The remaining F2F classes are lead by a variety of full-time and adjunct instructors with an array of teaching experience and educational background. To study the effect of changes, the three computer topics classes were divided into two groups, one class was the experimental group and the other two were the control group. The keyboarding classes were handled independently.

For the Fall 2003 semester, the instructors agreed that increased success without decreased rigor was the goal. The classes in the control group were handled with no significant difference from how the instructor handled them in previous semesters with regard to assignments, test pools and other assessments. It should be noted that all instructors involved were keenly aware of the issue at hand and that DE success was necessary to assure the continuation of the program. It is reasonable that the intensity of the instructors may have created a confounding variable affecting the results. Within the construct of this project, the following changes were enacted on the experimental group:

* Increased instructor monitoring and participation. This included an increase in the number of emails sent and a decrease in the response time of emails from students.

* Increased instructor participation in discussion threads.

* Changing the composition of and requirements for weekly homework assignments from a project-oriented assignment schedule where assignments were completed and then transmitted to the instructor, to a weekly chapter quiz where the quizzes were available until the end of the assignment week and then were no longer available.

* Increase in the number of web links to the Blackboard shell.

All other features of the classes remained the same. No changes were made to the assignment schedule, exam pools, or in the number of or composition of examinations.

Description of Changes

In the topics classes there was concern that a significant number of DE students were, for reasons unknown, simply disappearing from the classes. The instructors were of the opinion that compelling the student to be accountable for weekly assignments would have an effect on their continued participation. The idea behind this accountability was that if the student logged on to their Blackboard account more often, then they may be drawn into completing weekly assignments.

The aforementioned changes in the experimental topics class took three distinct forms. The first change was for the instructor to increase student monitoring and performance. This increase of student monitoring took two significant forms. One form was to drop the normal end-of-chapter exercises and assignments and replace them with short 10-question / 10-point quizzes. In the control group, students were assigned end of chapter questions or applications assignments which were completed and then transmitted to the instructor using Blackboard's Digital Drop Box. These assignments were then graded and the points awarded were then posted either weekly or monthly. In the experimental group, these assignments were dropped entirely and replaced with short multiple choice, true/false quizzes. These short quizzes were randomized out of the full exam pools and were set up to allow multiple attempts. Students were encouraged to take the quizzes often and if they scored low were able to retake the same quiz with newly randomized questions.

The quizzes were set up on an automatic timer that caused them to become unavailable over a succession of Mondays. Once the quiz was gone, it was gone. Thus, for example, while quiz 3 was available for the first three weeks of the class, it would vanish on the fourth Monday. This put an outside parameter on participation and created a sort of urgency to completing quizzes in a timely manner. It also allowed students so inclined to work forward quickly and get assignments "out of the way."

The second way that student monitoring was improved was by direct communication with students not participating in assignments. The instructor would regularly monitor participation in quiz assignments and when a student appeared to be falling behind, they would be sent a private email which pointed out the assignments missed and then gave encouragement and offered support. In two cases, "support" translated into direct, in-office, tutoring and explanation of concepts which rarely lasted more than several minutes. In all other cases, the student apologized for missing the assignments (rarely more than 30 points lost out of a semester possible of 900) and recommitted to the course. It should be noted that the instructor in the control group did likewise which may have had an effect on the increased success in those classes as well.

The third major change applied to the experimental class was to draw in current events directly applicable to the week's conversation in addition to the regular assignments. Blackboard has a section entitled "External Links" which allows the instructor to make web links available to the students within the class shell. During the semester, topical information, hardware and software releases, and other relevant events were placed in External Links. Then, "teaser" announcements were emailed to the students with instructions like "... check External Links now ..." to get students to sign on to Blackboard and into their accounts. There, grade information and other announcements had an opportunity to compel the student into continued participation.

Results

The project resulted in a significant increase in the success of the Business Division DE student without a change in the examination content or rigor. The results for the semester showed the Business Division DE success rate as only 5.5 points below the statewide average, or an increase of 15.2 points. The success rate in the computer topics classes increased from 40.0% to 47.6%. Additionally, the success rate in the keyboarding class rose from 31.1% to 52.6%.

Interestingly, during this time period and not included in the study, the Language Arts division also saw a remarkable increase in success from 13.6 points below statewide to 4.7 points above statewide across six offerings. The Math division suffered a decrease from 8.3 points below statewide to 23.0 points below across three offerings. This was accounted for by the results of two of their three DE classes.

Conclusions

Several conclusions can be drawn from this research. First, that in the California community college system, there is a significant difference in the success rates of DE verses F2F classes. Second, that difference is reflected to a lesser degree by the college in the project. Third, that difference can be directly impacted by the instructor. Fourth, that the combination of increased instructor participation along with increased student interaction had the most significant contribution to the student's success.

Discussion

When the distance education program was instituted at the College of the Sequoias, the resounding body of research indicated that DE and F2F courses were tending toward becoming identical in success, rigor, and acceptance by students, academia, and industry ("Employer and Student Perspectives ..." 1999). Research such as Russell's (n.d.) "no significant difference" gave COS faculty and administrators reason to believe that it was appropriate to directly compare the success rates of the DE classes with the success rates of the school as a whole. When those results were significantly different, it was decided to take a deeper look at the differences.

One important distinction in the COS study is that this research was conducted on junior college students. Much of the Russell-like research has been performed on graduate students (Beattie, 2002) or specialty programs (Anderson, Dougherty, Miller, Rentfro, & Roach, 2003) where the students have already achieved a certain level of maturity, discipline, and study habits; whereas in the junior college, those habits and disciplines are--in many cases--not fully developed. Further, COS is a commuter college as well (no on-campus student housing). As such these students are perceived to be "busy". Many of COS' students have jobs, family responsibilities, and are often primary care givers for their children or parents. The demographics show that the majority of the population is Hispanic females in their early 20's. In this culture, the issue of family care falls squarely on this student and education becomes a "luxury item". In the event that family responsibilities increase, it appears to be the education that gets dropped (Ginorio, 2000).

Therefore, this research has the significant impact of first acknowledging that there is a significant difference between the success rates in DE verses F2F classes. This difference is supported by this research and that of Nather (2003). Second, this research makes the point that it is inappropriate to compare the study discipline of the DE junior college student with that of the DE graduate student. Third, this study indicates that the success rates of the junior college DE class can be directly influenced by the instructor when that instructor practices guidance along with class management. The students at this level responded well to instructor encouragement and "handholding" and without diminishing the rigor of the course.

Endnotes

[1] Demographic and success rates are available at the California Community College Chancellor's Office Data Mart web site which is located at http://www.cccco.edu/divisions/tris/mis/reports.htm.

[2] COS data courtesy of the Dean of Instruction, College of the Sequoias.

References

Anderson, S., Dougherty, K., Miller, A., Rentfro, A.R., & Roach, S. (2003, June). Using educational technology and distance education to impact the shortage of baccalaureate prepared nurses. Online Journal of Nursing Informatics (OJNI), 7(2). Retrieved February 9, 2004 from http://eaa-knowledge.com/ojni/ni/7_2/anderson.htm.

Beattie, J., Spooner, F., Jordan, L., Algozzine, B., & Spooner, M. (2002). Evaluating instruction in distance learning classes. Teacher Education and Special Education, 25(2), 124-132.

Butler, D.W. (2004). Distance learning frequently asked questions. What you need to know about--distance learning. Retrieved on February 20, 2004 from http://distancelearn.about.com/cs/checkingouteschool/1/blfaqquality.htm

Donahue, S. (2002, November 15). Study pinpoints "tipping point" of successful programs. Distance education report, 6(22), 4

Employer and Student Perceptions of OUHK. (1999, September). Retrieved February 12, 2004, from The Open University of Hong Kong Center for Research in Distance & Adult Learning Web site: http://www.ouhk.edu.hk/CRIDAL/papers/employer.pdf

Ginorio, A. & Huston, M. (2000). !Si se puede! Yes, we can: Latinas in school. Retrieved March 17, 2004 from American Association of University Women Web site at http://www.aauw.org/research/latina.cfm

Nather, K. (2003 May). Distance Education Report. Chancellor's Office California Community Colleges Educational Services Division. v-vi.

Russell, T.J. (n.d.). The "no significant difference phenomenon". Retrieved on February 20, 2004 from http://teleeducation.nb.ca/nosignificantdifference/Schrum, L. (2002, November). Oh, what wonders you will see: distance education past, present and future. Learning and Leading with Technology, 30(3), 6-9, 20-1.

Scott, S.B. (2002). Coping with success: distance learning in Indiana higher education. Indiana Libraries, 21 (1), 2-5.

Waits, T. & Lewis, L. (2003). Distance education at degree granting postsecondary institutions: 2000-2001. Washington, DC: U.S. Department of Education, National Center for Education Statistics.

Wayne Machuca, College of the Sequoias, Northcentral University 

Machuca teaches computer topics and languages in the Business Division.