Effectiveness of web-based learning opportunities in a competency-based program.

This study examined adult students' progress in a competency-based online graduate program and their performance in various types of learning opportunities. Learned lessons and effective strategies were summarized. Students valued interaction with instructor as an important factor in online learning. New students had a better success rate in instructor-led online courses than in independent-study online courses. Adult students need modeling and scaffolding to be successful in an online environment.

**********

The Master of Arts in Learning and Technology (MLT), a competency-based online degree program at Western Governors University (WGU), has been running for over four years. Unlike traditional degree programs, the MLT measures students' progress by the number of assessments they have passed during a certain period. This poses a challenge for students to be highly motivated in order to complete any learning opportunities necessary to gain these competencies. The program culminates in a capstone project that requires students to identify an instructional problem and address it by designing, developing, and evaluating an instructional unit integrating technology into the instruction. When students work through each domain, they address the instructional problem from a particular angle (instructional design, research, use of technology or evaluation) by creating a project for the domain. Each project builds toward the final capstone, thus creating continuity from one project to the next. That continuity forces students to think globally about the program from the very beginning and make connections among the discrete domains. As a result, special demands are posed on learning opportunities; learning and instruction entail a drastic shift of the paradigm. In the context of competency-based education a learning opportunity refers to an instructional resource (such as an online distance course, online self-study modules, or textbook) that enables students to acquire required knowledge and skills. Some learning opportunities were more conducive to this shift than others and produced better results, which has kept us wondering about the following questions:

1. What types of learning opportunities have proved successful for competency-based online learning?

2. What supporting methods have worked well for our students learning at a distance?

3. Are all types of learning opportunities appropriate at different stages of the program?

4. What strategies have we used for selecting and designing learning materials?

Four years' practice has taught us many good lessons and provided us with insights into students' thoughts and needs in competency-based online learning. To better support our students, we felt a strong need to examine students' progress and their performance in our available learning opportunities, as well as to increase our understanding of our students as they interact with various learning opportunities and with us. The above four questions have guided our analysis.

PERSPECTIVES

The following educational perspectives have provided some guidelines in our practice and the analysis of the data.

Constructivist Learning Environments

The design of our program appears to fit very well what Jonassen (1999) described as Constructivist Learning Environments (CLEs) in that "the problem derives learning, rather than acting as an example of the concepts and principles previously taught. Students learn domain content in order to solve the problem, rather than solving the problem as an application of learning." (Jonassen, p. 218) One unique feature of our program is that each student is carefully guided in selecting an authentic instructional problem in her or his own environment that is "personally relevant or interesting to the learner." (Jonassen, p. 222) Once a student has identified a problem, the goal of the program is that the student will be able to produce a technology-based instructional unit to address that problem.

Seven Principles of Good Teaching Practices

Chickering's Seven Principles of Good Teaching Practices provide good guidelines for web-based instruction and students' satisfaction. Principle One, which is of particular importance to online learning, underscores the interaction between the students and teacher/expert. Frequent student-faculty contact is crucial in student motivation and involvement. "Faculty concern helps students get through rough times and keep on working." (Chickering & Ehrmann, 1996, pp. 3-6) This principle helps explain many of our students' comments about instructor-led courses.

Learning Orientations

Learning Orientation, which categorizes learners into transforming, performing, or conforming to resistant learners, helps us understand that not all learning opportunities are appropriate for all students at all times (The Training Place, 2001). Conforming learners prefer highly-structured learning environments and thus may be less successful online learners. Performing learners are typically self-motivated and self-directed only in areas that they value, but generally rely on external support. Transforming learners are highly self-motivated and independent and thus are likely to be successful online learners (Jones & Martinez, 2001, p. 2).

METHODOLOGY

Participants

Participants (N = 130) were our own mentees enrolled in the Master of Arts in Learning and Technology. All these students are adult learners, geographically scattered around the country. The majority of them are K-12 teachers; some of them are from corporate or higher education. Most of them are working full time with family responsibilities.

Data and Data Analysis

Data for the analysis were drawn from the WGU student information system as well as our notes on phone calls, e-mail messages, and review comments on students' written projects. We each examined our own data and compiled the results to be combined into a report. The program rubrics for the projects were used for judging a project quality. We employed categorical analysis of the data, and we entered course completion data, project completion data, and LOQ data into a spreadsheet for calculation of percentages. Categories and results are summarized next.

Types of Learning Opportunities

Learning opportunities in this study refer to the following:

1. Credit-bearing graduate courses that are offered over the Internet, including both instructor-led structured courses and independent, open enrollment courses.

2. Non credit-bearing graduate learning opportunities, including (a) instructor-led structured courses, (b) instructor-led open enrollment courses, (c) independent, open enrollment courses, (d) self-study modules, and (e) independent study with mentors.

RESULTS

Lessons Learned

Student-instructor interaction. Data indicate that students' satisfaction with a course was mostly related to rich student-instructor interaction. Most of our students' complaints about a course were associated with a lack of student-instructor interaction. "I could have done the readings myself." was a common comment we heard from dissatisfied students in an online course where interactions were minimal. Students were expecting not just interaction with peers, but more with their expert, the instructor. This expectation is in line with what has been found by Chickering and Ehrmann (1996) that adult students need as much student-instructor interaction as their younger counterparts and by Jiang and Ting (2000) that students' satisfaction online was significantly related to the amount of student-instructor interaction. In addition, in distance learning, students already missed the interaction with their instructor and with others in the class and that the immediacy behaviors of instructors have an impact on student satisfaction (Schulz, 2001; Arbaugh, 2001; Fielder, 2003).

An online course opens the door widely for communication seven days a week, 24 hours a day; thus, delivering a course online could demand more time than in a classroom. An instructor has to be well prepared to deal with the voluminous daily e-mails, discussion board messages, and students' projects; otherwise, s/he could easily be overwhelmed and fall short of students' expectations as well as her/his own. WGU sets interaction expectations for instructors of the online courses: (a) take active leadership in public online interactions and respond to students' individual questions within 24 hours (48 maximum); and (b) provide quick and constructive feedback on assignments and within days (a week maximum, depending on its complexity). Students' feedback indicates that when an instructor was too busy to show up in the discussion board, or keep up with feedback on students' projects, the course environment could become chaotic and students disgruntled.

Project-based versus quiz-based learning environments. As noted, our program fits well Jonassen's CELs in that all students have an authentic instructional problem to solve through the program. We enroll students only in courses that require a project; however, each course approaches the project in a different manner. Data indicate that students were less successful in online courses that emphasize chapter readings and quizzes without explicit guidance on their project, with a 50% project completion rate after the course. Students were found to continue to struggle with the project long after finishing such a course. In contrast, if an online course was project-oriented and required students to send in segments of a project as the course went along, most students were able to submit a satisfactory project shortly after the completion of that course. The completion rate was usually over 85%. This greatly impacts students' progress. If students can complete a domain project, they can quickly move on to the next domain, since most students found the domain objective and essay tests easier to pass than the projects.

Group work or collaboration. Course group work did not do well with our students. First of all, our program doesn't allow students to collaborate on WGU projects; therefore group work in courses is less valuable for our students. Second, these adults live in different time zones with scheduling difficulties. Group work tended to detract from the flexibility they value in their online, competency-based education. Requirements for group work in courses often turn out messy, because in no groups so far have all members actively participated equally.

Independent study versus instructor-led learning opportunities. The open enrollment policy of the university, students' individualized progress, and their busy schedules often create a need for flexibility of course schedules. To meet that need, open enrollment, independent-study courses are offered. Due to our inexperience at the early stage of the program, we allowed many new students to enroll in those courses that allowed students one year or half a year to finish. Results were astonishing: Those courses ended up with only 40-60% completion rate compared to the over 90% completion rate for instructor-led regular online courses. Many students tended to procrastinate in those courses and ended up requiring an extension or dropping from the course.

These shocking results prompted us to examine students' learning orientations. About two years ago, we implemented the Learning Orientation Questionnaire ([LOQ]; The Training Place, 2001). Data reveal that 65% of the 69 participants who filled out the LOQ survey fall in the category of performing learners, 28% transforming learners, and 7% conforming learners. Research about the LOQ indicates that performing learners are typically self-motivated only in areas that they value; otherwise, they reply on external support, for example, instructors (Jones & Martinez, 2001; Martinez, M., 2001). That could explain the high incompletion rate in our independent courses. Students' feedback on courses further indicate that most students were unable to progress through an independent course that gave them a year or half a year to finish and that they really needed deadlines to structure their learning. As one student put it, she did not realize the clock started to tick right after her enrollment and pretty soon the course expired. The same student took a structured course later. The instructor in that course aggressively engaged students in course activities and required students to submit segments of the project by deadlines. She completed the course with a very satisfactory project that needed minimal revisions.

Keys to Students' Success in Online Learning Opportunities

Different modes of learning opportunities at different stages of program. Students, especially new students, had a better completion rate in a structured course; therefore, providing a high degree of structure appears to be vital to students' progress. We tried to create a supportive environment for students at the very beginning of their program by putting them into instructor-led courses with intensive student-instructor interaction. Having gained some momentum in the program, many students would become more independent learners and could engage in more loosely structured study. It is believed that learners could move in and out of one learning orientation in response to negative or positive responses, conditions, results, and experiences (The Training Place, 2001). Our data indicate that engaging students in different modes of learning opportunities at different stages of their program has produced positive results.

Supporting methods. To enrich the experience of isolated online learning, we explored different alternatives to create learning communities such as real-time chat, discussion board, and phone conference (Eastmond & Jiang, 2003) and found that phone conferencing is convenient, efficient, and affordable. We have been regularly using the web-based conference service provided by freeconferencecall.com (http://www.freeconferencecall.com) for our conference calls. The conference call allows mentors to bring students together for live interaction with little cost and solve problems efficiently in a short time frame.

Cohort groups offer strong support for progress (Eastmond & Jiang, 2003). We have repeatedly received positive feedback about cohort groups. Some students commented that they could not have imagined working in the program alone. We have been forming cohort groups by encouraging students to enroll in the same course or work on the same domain for support.

Two years ago WGU started requiring all students to complete a four-week introductory course entitled Education Without Boundaries (EWB) in order to prepare them for success in our unique learning environment. During this course, students learn how to navigate through some web platforms and communication tools, manage their time, and use our online library services. Students are also asked to write a 4-5 page Problem Statement paper that must include 5 references and be written in APA style. Passing the course is a requirement for continued matriculation at the university. Our data revealed that students who took EWB had greater confidence working in our online learning opportunities, took greater efforts working on the projects, and as a result, progressed faster than those earlier-enrolled students for whom this was not a requirement.

Web-based modules. To best meet the needs of our students WGU faculty created web-based self-study modules and supported these by monthly phone calls and daily e-mails. Course materials developed in course management systems such as WebCT and Blackboard could be buried under numerous things such as Introduction, Announcements, Staff information, Communication, Discussion Board, External Links, Tools, Virtual Chat, Student Tools, and so forth. Students repeatedly reported disorientation and frustrations in those environments. Therefore we set up the modules in a simple web-based environment, and focused these self-study modules only on the content and projects. Within each topic, we provided key questions with key points to explicitly guide students through the assigned reading materials and the development of their project. A big chunk of a module was devoted to guiding students in developing their project with step-by-step instructions, guiding questions, model samples, and section-by-section sample analysis. The module design strives for simplicity, clarity, and conciseness with explicit coaching. The goal is to offer students "a course with clear, comprehensive ... content trimmed to fit the students' time budget" (Moore, 1998, p. 5), as well as the type of coaching that students might receive in a traditional classroom setting. Immediate response to students' questions was provided through e-mail in addition to the monthly phone conference for each module. These modules have been effectively used by students as demonstrated by their successful completion of the target domain.

Modeling and scaffolding by using samples and project templates. A competency-based program requires students to reach the required competencies, which has placed a great challenge on many students who might otherwise just receive a B or C for their written projects. Competency-based learners generally need to reach a course grade above B+ to pass their WGU project assessments, a difficult task for both mentors and students. We found that no matter how detailed the instructions or guidelines were for a project, students would miss some sections or important points, thus creating numerous frustrations on both sides. According to Jonassen (1999), in CELs, the mental models that "naive learners build to represent problems are often flawed. They often misattribute components of the problem or incorrectly connect them" (p. 234). Jonassen recommended that the project or problem requirements are clearly communicated and students are provided with an example of the desired performance as modeling. Therefore, for each domain project, we have provided 2-4 samples as model projects. To "point out each of the discrete actions and decisions involved in the performance, so that the learner is not required to infer missing steps" (Jonassen, p. 232), we have provided scaffolding or coaching by using project templates outlining the key points for students to address or a set of guiding questions for students to answer in their projects. The implementation of a template has obviously achieved its objective. Data indicate that projects developed using a template were usually more complete, meeting the requirements at a higher level for further improvement.

CONCLUSION

We found that problem-based courses with intensive student-instructor interaction were more conducive for our competency-based online distance learners, especially in the early stages of their programs. Later, web-based modules that provide substantial coaching and scaffolding, supported by phone conferences and timely email responses are as effective once students have gained greater confidence and independence. Sample projects and templates provide necessary and important models and scaffolding for competency-based distance online learners to reach the required competencies.

References

Arbaugh, J.B. (2001). How instructor immediacy behaviors affect student satisfaction and learning in web-based courses. Business Communication Quarterly, 64(4), 42-53.

Chickering, A. W., & Ehrmann, S. C. (1996, October). Implementing the seven principles: Technology as lever, AAHE Bulletin, (pp. 3-6).

Eastmond, D., & Jiang, M. (2003, August 14). Ensuring program success through learning communities. Paper presented at the 19th Annual Conference on Distance Teaching and Learning at Madison, Wisconsin and published in the conference proceedings.

Fielder, S. K. (2003). Faculty attitudes toward student satisfaction with online courses. Research proposal submitted as a partial fulfillment for the Masters of Arts in Learning and Technology program at Western Governors University.

Jiang, M., & Ting, E. (2000). A study of factors influencing students' perceived learning in a web-based course environment. International Journal of Educational Telecommunications, 6(4).

Jonassen, D. (1999). Designing constructivist learning environments. In C. Reigeluth (Ed.), Instructional-design theories and models: A new paradigm of instructional theory (Vol. II, 215-239). Mahwah, NJ: Lawrence Erlbaum.

Jones, E. R., & Martinez, M. (2001). Learning orientations in university web-based courses. Paper presented at WebNet, 2001, Orlando, Florida. Retrieved January 22, 2006, from http://www.tamucc.edu/~ejones/papers/webnet01.pdf

Martinez, M. (2001). Key design considerations for personalized learning on the web. Educational Technology and Society 4, 26-40.

Moore, M.G. (1998). Introduction. In C.C. Gibson (Ed.) Distance learners. Madison, WI: Atwood Publishing.

Schulz, C. (2001). Surveys of distance learning in the Virginia Community College system. Inquiry, 6(2). Retrieved January 22, 2006, from http://www.vccaedu.org/inquiry/

The Training Place (2001). Learning orientation questionnaire. Retrieved January 22, 2006, from http://www.trainingplace.com/loq/loqinfo.htm

MINGMING JIANG, SYDNEY PARENT, AND DAN EASTMOND

Western Governors University, USA

[e-mail]

[e-mail]

[e-mail]