Engaging and assessing students in online courses.

Abstract

The current article discusses how to capitalize on the tools in online course management programs (e.g., WebCT, Blackboard, etc.) in order to engage students and assess student learning and performance. By using these tools appropriately, a collaborative learning community may be formed in an online or hybrid course.

Introduction

The usability of instructional technology has put a new spin on education. As instructional technology has been rapidly implemented into the educational system, institutions of higher learning have rushed to integrate instructional technology into their academic programs. This has allowed them to be more competitive and offer students a choice of convenient course offerings (e.g., web-enhanced, online, and hybrid courses). Also, instructional technology is often misperceived as a magic bullet which will increase student performance and learning. As a result, academic institutions have invested a great deal of time, effort, and money into the implementation, maintenance, and support of instructional technology.

However, little research has been conducted to investigate how instructional technology can be used more effectively. Many questions have yet to be answered. For example, in what type of environments is instructional technology most effective? How can instructional technology be used to engage students and increase learning? How can instructors assess learning in courses using instructional technology? How can instructors modify their instruction using instructional technology in order to be more effective? Overall, how will the use of instructional technology as a means to teach students and convey course material redefine the role of educators and reshape classroom learning experiences? This paper will take an important first step in discussing how instructional technology can be used to engage students and how student learning in a technology-based course should be assessed.

Effectiveness of Instructional Technology

The research that has been conducted to evaluate the effectiveness of instructional technology has yielded mixed findings. Some studies have found that integrating technology into the classroom creates an effective learning environment which improves student learning (see Cronin, Meadows, & Sinatra, 1990; Funkhouser, 1993; Hutchens, 2004b, Hutchens, 2005; Luna & McKenzie, 1997; Miller & Hutchens, under review; Sammons, 1995; Sherry, Jesse, & Billig, 2002; Traynor, 2003; Zack, 1995). For example, Funkhouser (1993) found that high school mathematic students which received computer-assisted instruction using problem-solving mathematics software demonstrated significant increases in mathematic knowledge and problem-solving skills as compared to students which did not receive computer-assisted instruction. The students in the computer-assisted condition also reported having more positive attitudes about mathematics and about themselves as learners (Funkhouser, 1993). Moreover, studies measuring pretest-posttest gains have yielded significant gains when using instructional technology to teach secondary school students (Cronin, Meadows, & Sinatra, 1990; Sherry, Jesse, & Billig, 2002; Traynor, 2003). Specifically Traynor (2003) demonstrated that students in various programs (i.e., special education, non-English proficient, limited English proficient, and regular education) all made significant pretest-posttest gains when using a computer-assisted instruction program called CornerStone. Other studies have shown that college students benefit from the use of instructional technology as well as secondary students (George & Sleeth, 1996; Hutchens, 2004b, Hutchens, 2005; Luna & McKenzie, 1997; Miller & Hutchens, under review; Sammons, 1995; Zack, 1995). Survey data collected by Sammons (1995) indicated that the majority of liberal arts majors reported that multimedia presentations were much more effective than presentations using an overhead projector and a chalkboard. Also, Zack (1995) found that electronic messaging significantly improved quality of instruction. Luna and McKenzie (1997) demonstrated that community college students' performance improved when using multimedia technology as a form of instruction as compared to traditional lecture. They also found that 80% of the students agreed that using a multimedia (i.e., CD-ROM) was a positive experience in regard to ease of use and contribution to the course.

However, George and Sleeth (1996) measured business majors' performance and attitudes in business course which utilized instructional technology and found an increase in student satisfaction but no increase in student performance. They indicated that instructional technology motivated students by increasing enjoyment, interest, and learning, but student motivation did not significantly increase test and paper grades (George & Sleeth, 1996). Also, Hutchens (2004b) investigated student performance (i.e., test grades) in General Psychology courses which were either taught using no technology (i.e., chalkboard lecture), technology-assisted using PowerPoint and WebCT with complete course lecture notes, or technology-assisted using PowerPoint and WebCT with partial course lecture notes. Hutchens (2004b) stated that partial notes are effective because they allow for class lectures to flow smoothly, increase students' expectations about the course material being covered, provide students with an organized framework of notes to build on and modify, and capitalize on a powerful memory phenomenon called the generation effect (i.e., generated material is remembered better than read material) (see Slamecka & Graf, 1978; Gardiner & Rowley, 1984; Marsh, Edelman, & Bower, 2001; McElroy & Slamecka, 1982).

However, other studies have not shown a benefit in student performance (see Avila, Biner, Bink, & Dean, 1995; Branton & Lee, 2003; Garrett, 1995; George & Sleeth, 1996; Guy & Frisby, 1992; Hutchens, 2004a). For example, in a within-participants design, student learning was not increased when college students were taught using computer-generated graphics (CGGs) as compared to using traditional visuals (Avila, Biner, Bink, & Dean, 1995). In fact, when students were in the CGGs condition, they reported having less positive attitudes concerning the helpfulness of the material and the instructors' preparation and organization than when they were in the control condition. This may have been because the instructors were not as familiar with using CGGs as they were with using the traditional visual materials. Also, Guy and Frisby (1992) found that the performance of pre-nursing and allied-health college students in a gross anatomy course did not differ significantly when using an interactive videodisc computer lab as compared to a traditional demonstration lab.

Furthermore, Branton and Lee (2003) demonstrated that both undergraduate and education graduate students did benefit from the use of PowerPoint lectures versus traditional lectures. Moreover, similar to George & Sleeth (1996), Hutchens (2004a) found that, even though students reported being more satisfied with General Psychology courses using instructional technology, student performance was significantly lower in web-enhanced, PowerPoint lecture (complete-note) courses as compared to courses taught using traditional chalkboard lecture. As discussed earlier, Hutchens (2004b) demonstrated that student performance was most likely lower in the technology condition due to giving students complete lecture notes. Thus, in light of these mixed findings, it is evident that more research is needed to determine effective uses and conditions for instructional technology, and appropriate methods for assessing student learning.

Technology-Based Courses

There are several different technology-based courses: Web-enhanced, online, and hybrid. A web-enhanced course meets every scheduled class meeting but also has a web (or technology) component such as an online course management program (e.g., WebCT and Blackboard) and presentation software (e.g., MS PowerPoint). Students may prefer web-enhanced courses over online courses because they want to have an increased sense of belonging to the class, direct guidance from the instructor, face-to-face instruction, or feedback from the instructor. Online courses do not meet in a physical classroom; instead all interactions typically take place through a dedicated program or email. Online courses can be synchronous or asynchronous. Students in synchronous online courses may be physically separated but meet virtually at the same time through video-teleconferencing or online chat-rooms. Asynchronous online courses allow students to work and meet at separate times at their convenience anywhere they have Internet access through the use of online testing, assignments, discussion boards, and email. Because of the convenience of asynchronous courses, students may prefer asynchronous courses over synchronous courses. Finally, a hybrid course combines the elements of web-enhanced and online courses. Students physically meet and work in class half of the time and virtually meet and work online half of the time. Students may prefer hybrid courses because they may receive instruction and feedback from the instructor in a face-to-face environment, but also have the convenience of working online.

Engaging and Assessing Students

Technology-based courses cannot be taught and assessed the same way as traditionally-taught courses. Thus, depending on course type, instructors should appropriately modify their instruction techniques and assess learning in different ways in order to be effective. A major problem with most online courses is that 30 percent of students drop out (Miller, King, & Hutchens, unpublished manuscript; Moore & Kearsley, 1996). This may be due to instructors failing to modify their instructional and assessment techniques appropriately for technology-based courses. Thus, students may drop out due to anxiety, confusion, frustration with technology, lack of timely feedback, lack of any feedback, lack of human contact, feelings of isolation (Miller & Hutchens, under review), forgetting deadlines, and procrastination. To eliminate the above problems, online course instructors should strive to create a collaborative learning community within their courses (Miller, King, & Hutchens, unpublished manuscript). These courses should be designed in such a way as to establish communities of learners, facilitate social interaction, active participation, cooperation, exploration, and collaboration (Hiltz, 1997; Miller, King, & Hutchens, unpublished manuscript; Spitzer, 1998). Instructors must also strive to enhance their personal touch in the course in order to increase a student's sense of belonging (Miller, King, & Hutchens, unpublished manuscript). Establishing a collaborative learning community should increase students' sense of human contact and therefore reduce feelings of isolation, confusion, and anxiety.

Also, online courses should be designed so that students must be active participants, become more responsible for their own learning, and have or develop good communication skills (Miller, King, & Hutchens, unpublished manuscript). Students cannot passively listen to course lectures and rely on the instructor to make sure they learn the material as some do in traditional courses. Instructors must understand that teaching an online course is quite different from teaching a traditional course. That is, the instructor's role has shifted significantly from the one primarily delivering course material in a traditional course to that of guide or mentor to students in their learning of the material (Bourne, McMaster, Rieger, & Campbell, 1997; Miller, King, & Hutchens, unpublished manuscript). Therefore, online courses must be collaborative, student-centered, self-regulated, self-directed, and constructivist (Miller & Hutchens, under review). Moreover, students must understand that the most important predictor of success in an online course is engaging in self-directed learning. The student should understand that an online course is more difficult than a traditional course due to the increased amount of responsibility that falls on the student. The student must carefully read the textbook and online resources in order to gain the majority of course information. The student must also manage his or her time effectively in order to learn in an online course environment. Finally, a student cannot afford to procrastinate in an online course. Instructors can accomplish the aforementioned goals of establishing a collaborative learning community and encouraging active student participation by using appropriate techniques for engaging students and assessing student learning in online courses. For example, instructors can use the following tools in most online course management programs (e.g., WebCT, Blackboard, etc.) in order to accomplish these goals and, as a result, increase student learning and satisfaction while increasing completion rates. Online discussion boards, email, assignments, quizzes/tests, group presentations, progress tracking, and grade books are course management tools that can be used effectively to engage students and assess student learning in online courses.

First of all, instructors should capitalize on the discussion board tool. The discussion board tool allows for students to participate in an online asynchronous discussion. Students can post replies to instructor-and student-initiated topics. Instructors may have students use the discussion board to discuss course material and answer specific questions. The discussion board generates a threaded virtual paper trail that students can read and reflect on and the instructor can grade based on quality of content. The discussion board is an excellent tool to help create a collaborative learning community which promotes student-student and instructor-student interaction. Also, if used appropriately, the discussion board allows for student collaboration, instructor and student feedback, writing development, and critical thinking. As a result, online students will engage in active participation and collaboration, and have an increased sense of belonging to the class.

Also, the online assignment and quiz/test tools can be utilized to encourage active participation and give timely and informative feedback. Assignments and quizzes/test can be composed of a combination of objective and subjective questions. In WebCT and Blackboard, objective questions can be graded automatically and the student can receive feedback from the instructor for each question immediately. Subjective questions may be graded easily by the instructor within the course management program with the ability to give detailed and thorough feedback. Importantly, the quiz/test tool can be set to randomly select a number of questions from a large test-bank based on certain criteria. The instructor can also adjust the tool to allow students to take quizzes multiple times and record the highest, last, or average grade. The quiz/test tool can also be used to release quizzes/tests to students based on multiple criteria. Also, as with the discussion board tool, a virtual paper trail is maintained in the course management program for the instructor's and student's use. Students can reflect on and study their assignment and quiz/test results at their convenience throughout the semester in order to prepare for other tests or a comprehensive final. In online asynchronous courses, instructors must realize that their quizzes/tests are potentially open-book unless they have students take quizzes/tests in proctored computer laboratories. However, an instructor can control open-book online quizzes by setting a short quiz duration. For example, if the quiz is 25 multiple choice questions, the instructor can set the quiz duration at 20 minutes. Thus, the student must submit all answers before the deadline in order to earn credit. By setting a short quiz duration, a student must read and become familiar with information in the textbook before taking a quiz/test. Otherwise, the student will run out of time while searching for information. The instructor should also use at least application level questions so that students must understand the material to answer the questions instead of just searching for a definition in the textbook.

Most online course management programs also have a group presentation tool so that students in large online classes can be assigned to small groups in order to work on group projects, assignments, papers, and presentations. Groups can also be designated in the discussion board so that group members can discuss their projects online. The group presentation tool helps establish a collaborative learning community by facilitating a sense of belonging to a small group, increasing social interaction and collaboration, and sharing knowledge.

Moreover, most online course management programs have a tracking tool that instructors can use to evaluate a student's progress in the course (e.g., course hits, items read, number of posts, time spent on quizzes, etc.), as well as the progress of the entire class as a whole. This is valuable data that can be used to assess student engagement, learning, and the effectiveness of instructional technology. For example, Hutchens (2004b) demonstrated that the number of items read and number of course hits are positively correlated with student performance. The instructor can also make this valuable tracking data available to the students so they are aware of their progress. In this way, students may reflect on their course performance and modify their productivity in order to increase learning and performance.

Also, course management programs allow each student to view his or her row of the instructor's grade book. In this way, there is no discrepancy between the instructor's assigned grade and the grade the student thought he or she earned. A student can also view descriptive statistics for each assignment or quiz/test in order to judge how he or she compared to other students in the course. Also, the instructor can release all assignments, quizzes, and tests to the student's grade book so that the student can be more aware of what he or she has completed, facilitate expectations about future course work, and realize the value of each assignment or quiz/test. A collaborative learning community may be established by the grade book tool by allowing for timely feedback and communication. In this way, the grade book tool makes it possible for each student to be fully aware of his or her performance in the course.

Finally, instructors can use the email tool in the course management program to easily email the entire class, a group of students, or a single student. Also, most email tools may be adjusted so that all email messages are forwarded to the student's or instructor's personal email account from the course management program. This feature allows the instructor and student to receive course communication without logging in to the course management tool several times a day. It is important to mention that course management email tools are also more dependable than using students' personal email accounts because students' accounts may often be full, unreliable, or changed. Also, the email tool may be used to send and receive course information, attachments, reports, and assignments. A collaborative learning community can be created by using this tool because it provides a means for social interaction, collaboration, communication, and timely feedback. In order to create a sense of belonging and facilitate a personal touch, it is very important that instructors communicate and quickly provide feedback to each individual student instead of sending email messages to the entire class.

Conclusion

Online course management programs provide many useful tools (e.g., online discussion boards, email, assignments, quizzes/tests, group presentations, progress tracking, and grade books) that can be used to create a collaborative learning community in order to engage students, provide students with information, and assess student learning. Furthermore, qualitative and quantitative data from course evaluations and surveys have indicated that most students tend to be more satisfied in technology-based courses than in traditional courses (George & Sleeth, 1996; Hutchens, 2004a; Luna & McKenzie, 1997; Sammons, 1995; Zack, 1995). Out of over 400 undergraduate students in General Psychology courses surveyed, the majority of the students reported that they enjoyed using online discussions, assignments, simulations, demonstrations, PowerPoint shows, and the ability to check their grades using the course management program, WebCT (Hutchens, 2004a). The same students also reported that they appreciated the fact that course material was available anytime, anywhere (Hutchens, 2004a).

In conclusion, as discussed earlier, the data concerning the effectiveness of instructional technology is rather mixed. It may be the case that presenting instructional material to students using technology is just another medium (e.g., web pages, PowerPoint shows, transparencies, chalkboards) for presenting information which may or may not lead to significant increases in student performance and learning. On the other hand, it is most likely the case that instructional technology is an effective and innovative tool for conveying information and increasing student learning. However, it is just that ... a tool! One must learn how to use the "tool" in an appropriate manner in order to affect student outcomes. That is, it is important that educators do not use instructional technology just for the sake of using it. Instead, educators should strive to develop innovative teaching strategies that increase student learning and comprehension. If the use of instructional technology can help achieve this goal, then it should be considered for implementation. Instructors must realize that technology-based courses (i.e., web-enhanced, hybrid, and online), have be taught and assessed in different ways as compared to traditionally-taught courses. As discussed earlier, instructors can help accomplish this task by appropriately using the available online course management tools to develop a collaborative learning community which facilitates social interaction, active participation, cooperation, exploration, and collaboration within their courses (Hiltz, 1997; Miller, King, & Hutchens, unpublished manuscript; Spitzer, 1998). Thus, future research should examine how online course management program tools may be optimized to create a collaborative learning community in order to promote student engagement and effectively assess student learning.

References

Avila, R. A., Binet, P. M., Bink, M. L., & Dean, R. S. (1995). Course materials presentation using video-based technologies: An evaluative study of college student performance and attitudes. Psychology in the Schools, 32 (1), 38-45.

Bourne, J. R., McMaster, E., Rieger, J., & Campbell, J. O. (1997). Paradigms for online learning: A case study in the design and implementation of an Asynchronous learning networks (ALN) course. Journal of Asynchronous Learning Networks, 1 (2), 38-56.

Branton, C. B., & Lee, S. P. (2003). Student learning compared through the use of student PowerPoint presentations and traditional teaching methods, Delta Education Journal, 1, 11-15.

Cronin, H., Meadows, D., Sinatra, R. (1990). Integrating computers, reading, and writing across the curriculum. Educational Leadership, 48 (1), 57-63.

Funkhouser, C. (1993). The influence of problem-solving software on student attitudes about mathematics. Journal of Research on Computing in Education, 25 (3), 339-347.

Gardiner, J. M., & Rowley, J. M. C. (1984). A generation effect with numbers rather than words. Memory and Cognition, 12, 443-445.

Garrett, R. L. (1995). Computer-assisted instruction in 2-year colleges: Technology for innovative teaching. Community College Journal of Research and Practice, 19 (6), 529-536.

George, G., & Sleeth, R. G. (1996). Technology-assisted instruction in business schools: Measured effects on student attitudes, expectations, and performance. International Journal of Instructional Media, 23 (3), 239-245.

Guy, J. F., & Frisby, A. J. (1992). Using interactive videodiscs to teach gross anatomy to undergraduates at the Ohio State University. Academic Medicine, 67 (2), 132-133.

Hiltz, S. R. (1997). Impacts of college-level courses via asynchronous learning networks: Some preliminary results. Journal of Asynchronous Learning Networks, 1 (2), 1-19.

Hutchens, S. A. (2004a). Teaching psychology using technology: An investigation of student performance, attendance, and satisfaction. Delta Education Journal, 1 (2), 5-15.

Hutchens, S. A. (2004b). Investigating appropriate uses of instructional technology: Is technology-assisted instruction effective? Delta Education Journal, 2 (1), 9-19.

Hutchens, S. A. (2005). Evidence of a successful partnership: Universities working closely with area schools. In S. Rhine & M. Bailey (Eds.), Integrated technologies, innovative learning: Insights from the PT3 program (pp. 59-66). Eugene, OR

Kroder, S. L., Suess, J., & Sachs, D. (1998). Lessons in launching web-based graduate courses. T.H.E. Journal, 25 (10), 66-69.

Luna, C., & McKenzie, J. (1997). Testing multimedia in the community college classroom. T.H.E. Journal, 24 (7), 78-81.

Marsh, E. J., Edelman, G., & Bower, G. H. (2001). Demonstrations of a generation effect in context memory. Psychonomic Society, 29 (6), 798-805.

McElroy, L. A., & Slamecka, N. J. (1982). Memorial consequences of generating nonwords: Implications for semantic-memory interpretations of the generation effect. Journal of Verbal Learning and Verbal Behavior, 21, 249-259.

Miller, T. W., & Hutchens, S. A. (under review). Best practices and effectiveness in teaching psychology through distance education. Teaching of Psychology.

Miller, T. W., King, F. B., & Hutchens, S. A. (unpublished manuscript). 21st century teaching technology: Pedagogy in teaching psychology.

Moore, M. G, & Kearsley, G. (1996). Distance Education: A systems view. San Francisco: Wadsworth Publishing Company.

Muirhead, B. (2000). Enhancing social interaction in computer-mediated distance education. Educational Technology & Society 3 (4), 1-11.

Palloft, R. M., & Pratt, K. (2001). Lesson from the cyberspace classroom: The realities of online teaching. San Francisco: Jossey-Bass Publishers.

Sherry, L. Jesse, D., & Billig, S. H. (2002). Creating a WEB of evidence of student performance in a technology-rich learning environment. International Journal of E-Learning, 1 (1), 33-42.

Slamecka, N. J., & Graf, P. (1978). The generation effect: Delineation of a phenomenon. Journal of Experimental Psychology: Human Learning and Memory, 6, 592-604.

Smith, G. G., Ferguson, D., & Caris, M. (2001). Teaching college courses online vs. face-to-face. T.H.E. Journal, 28 (9), 18-26.

Spitzer, D. R. (1998). Rediscovering the social context of distance learning. Educational Technology, 38 (2), 52-56.

Traynor, P. (2003). Effects of computer-assisted-instruction on different learners. Journal of Instructional Psychology, 30 (2), 137-143.

Scott A. Hutchens, Delta State University, MS

J. Reid Jones, Delta State University, MS

Darlene E. Crone-Todd, Delta State University, MS

Heidi L. Eyre, Delta State University, MS

Dr. Hutchens is an associate professor in Psychology, Dr. Jones is a professor in Psychology, and Dr. Crone-Todd and Dr. Eyre are assistant professors in Psychology at Delta State University in Cleveland, Mississippi.