Maximizing learning from rehearsal activity in web-based distance learning.

Faculty teaching distance courses continuously seek ways to maximize learning for students. Two practice with feedback strategies were examined in this study for their impact on students' achievement and attitudes in an upper-division university level web-based course. One format contained higher structure and dialogue creating lower transactional distance for rehearsals and the other was lower structure and dialogue creating higher transactional distance for learners. Forty-five upper division students enrolled in a web-based distance course were blocked by major and randomly assigned to one of two rehearsal treatments: (1) practice tests with correct answers displayed below the tests for comparison or (2) interactive practice tests with computer-generated feedback for correct answers and percentage correct test scores. Results indicate that the students taking interactive quizzes in the lower transactional distance format (computer-generated item feedback and test scores) performed significantly better on the final examination (p = .01) and rated the practice and feedback activities as more relevant (p = .03) than students who were free to view the questions and answers on their own.

**********

Web-based distance courses are increasing on university campuses throughout the world, and faculty members strive to ensure that quality teaching and learning is maintained for remote students. Numerous studies have demonstrated that different instructional delivery formats are not predictive of learning effectiveness; rather, the key to ensuring the quality of distance learning lies in the application of proven, theory-based instructional strategies for course delivery (Clark, 1983; Dick, Carey & Carey, 2005).

Cognitive psychologists conclude that one essential element of an effective instructional strategy is the practice with feedback component (Gagne, 1985). Principles of effective practice include that it be: (1) located in reasonable proximity to instruction, (2) of appropriate scope and breadth for memory, (3) congruent with the skills included on posttests, and (4) non-threatening to learners. Additionally, effective feedback should provide evidence of the correctness of responses, contain explanations when correct answers are not obvious (Gagne, 1985; Dick, Carey & Carey, 2005), and it should be prompt (Dihoff, Brosvic, Epstein, & Cook, 2004).

Practice and feedback traditionally were in-class exercises or homework assignments that were scheduled and supervised by the instructor. In web-based courses, however, it is possible for students to manage their own instruction and pace, and to decide for themselves whether and how they will use the practice and feedback materials provided within the instructional site. Moore's theory of transactional distance (1996) provides some insight into varying levels of structure and dialog that impact transactional distance making it more or less appropriate based on learners' maturity.

Designing practice and feedback materials beyond the basic learning principles to include considerations for varying levels of transactional distance may make rehearsal more effective in distance courses. The structure and dialog dimensions of Moore's theory are summarized in Table 1 (Carey & Gregory, 2002; Dick, Carey, & Carey, 2005). Essentially, less mature learners who have not developed strong metacognitive strategies require greater structure in intra-course sequencing and materials as well as more opportunities for dialogue (personalized communication) with the instructor and/or a technology substitute. For less mature learners, Moore's theory would advise instructional strategies that require participation in practice activities (high structure) and feedback that is personalized and delivered to the student (high dialogue). The value of course structure and student interaction is also supported at the intuitive level in many guides for distance learning practitioners (e.g., Chute, Thompson, & Hancock, 1999; Williams, Paprock, & Covington, 1999; Willis, 1993).

From the perspective of student attitudes, practice and feedback material could be viewed on a continuum from low stakes to high stakes in terms of perceived consequences to students, and the higher the stakes, the more motivated students are to achieve higher scores (Wise & DeMers, 2005). High transactional distance practice (scroll-down format) would be categorized as having low stakes consequences from the learners' perspective while practice in a more-formal, computer-scored format would be considered higher stakes because the instructor would have knowledge of completion and performance. The learner who is conscious of grades and made anxious by the possibility of a wrong answer on a test may feel less comfortable with the low transactional distance (computer-scored) practice format.

[TABLE 1 OMITTED]

Also related to students' attitudes, Keller (1987) synthesized four variables from academic motivation theory into the ARCS model which includes additional principles for instructional design that would help designers ensure students' Attention to instruction, perceived Relevance of the instruction to them, Confidence in their ability to succeed, and personal Satisfaction with their accomplishments. It is likely that varying practice and feedback formats would affect students' academic motivation differently. Related to this study, the location of the practice and feedback within the instruction may influence the extent to which it gains and holds students' attention. Likewise, the format of practice and feedback activities, for example, low transactional distance of read, respond, evaluate, and reflect by scrolling versus high transactional distance by completing and submitting for scoring an objective-format quiz, may influence students' perception of the relevance of the practice for their exam preparation, their confidence in their answers, and their satisfaction with their scores.

From an instructional design perspective, it is important to examine practice and feedback strategies using differing degrees of structure, dialogue, and transactional distance within web-based instruction. The purpose of this study was to examine the effect of varying the transactional distance proximity (Moore & Kearsley, 1996; Carey & Gregory, 1999) of the practice and feedback component by comparing a higher transactional distance scrolling method to a lower transactional distance practice quiz format. Further, the study will examine the effect that varying the transactional distance proximity has on students' academic motivation as measured by the Academic Motivation Profile (Carey, Carey, Dedrick, Kushner, & Greenbaum, 1994).

METHODS

Sample

This study was conducted in a doctoral/research-extensive university (Carnegie, 2005) located in the southeast. The university has over 35,000 students enrolled in traditional as well as blended and total distance courses and degree programs. Participants in this study included all 45 students enrolled in an upper-division undergraduate course in principles of assessment for learning taught within the Department of Educational Measurement and Research. Participants all self-selected the web-based course format.

Instructional and Assessment Materials

The syllabus, all instructional materials, and all assessments were included within the university's web-based course management system. The study spanned the last four weeks of the semester and covered four instructional goals. Instruction for each of the four goals contained learning objectives, information presentation, examples, and demonstrations as well as two multiple-choice practice tests with feedback. One practice test covered lower-order concepts and vocabulary from the textbook and web sessions and the other covered higher-order skills such as data synthesis, calculation, interpretation, and evaluation for a total of eight practice tests with feedback.

The low structure, low dialogue, greater transactional distance practice format. The practice and feedback was placed last within the instructional segment for each goal. Students received instructions to study the questions and answers and to complete the quizzes by writing their answers to the questions on their own paper. At the end of the questions, students were instructed to compare the answers they had written for each question with those in the feedback section which followed the questions. Students scrolled down to the feedback section where the questions and answers were repeated, the correct answer was highlighted, and a brief, one- or two-sentence explanation was provided beneath the question. In this low structure model, students were free to choose whether they would actually answer the questions, briefly ponder them, or simply scroll forward to the feedback to browse comfortably through the questions, answers, and explanations. The degree of their engagement with the problem within each question and the instructional utility of the feedback depended solely on their level of learning autonomy.

The high structure, high dialogue, low transactional distance practice format. The location of the practices within instruction, the same multiple-choice questions, and the same one- or two-sentence explanations were also included in this format. The only difference was that students could only access the practice tests and feedback using the university web-based course management's test administration function. After they answered all the questions, they submitted their answers for instantaneous computer scoring. The program returned information about whether each item was correct along with the brief explanation and a total test percentage correct score. In this structured and interactive model, the transactional distance was much less. Students encountered the problems in a more formal context, needed to answer the questions in order to obtain the feedback, were confronted with whether they were correct, and were provided with a score describing the overall quality of their work.

Achievement posttest. The achievement posttest consisted of a 40-item multiple-choice final examination with items that measured only higher-order skills such as using given data sets to synthesize, calculate, interpret, evaluate, and communicate results. The posttest was administered to both groups in a proctored computer laboratory on campus on the same day. The test was developed using principles of criterion-referenced test design, and the internal consistency reliability index was good for this instructional context.

Academic motivation. The attitude instrument contained nine items related to students' interest in various online instructional components, nine items on the relevance of the rehearsal materials for helping them prepare for course examinations and their professional roles, nine items related to their confidence in performing the skills learned, and nine items related to their personal satisfaction with their own efforts in the course. The items had a 4-point response format with a rating of I representing no feelings of interest, relevance, confidence or satisfaction to a high rating of 4 representing very high levels. The instrument was based on the Academic Motivation Profile which has demonstrated quality psychometric characteristics using internal consistency reliability (r = .89 - .98) and confirmatory factor analyses in previous studies with the same course content and similar students (Carey, Carey, Dedrick, Kushner, & Greenbaum, 1994).

Procedures

Prior to the beginning of the term, students were stratified by major and then randomly assigned to one of two course management websites containing either the low structure and dialogue (high transactional distance) practice and feedback format or the high structure and dialogue (low transactional distance) format. For the entire term, both groups of students studied independently within their assigned website, rehearsed using their assigned practice and feedback format, communicated with their faculty member via email, and returned to campus only for the posttest exams. Students in both groups had the same textbook, instruction, and instructor; followed the same course syllabus and study schedule; and took the same computer-based posttest examinations and surveys. Group performance was compared using t-tests.

RESULTS AND CONCLUSIONS

Achievement

The means, standard deviations, and reliabilities for the final examinations are included in Table 2. The low transactional distance (computer-scored) practice and feedback group performed significantly better (p = .01; d = .65) than the high transactional distance (self-scored) practice and feedback group on the final examination.

Attitude

The means, standard deviations, and reliabilities for the four attitudinal variables are presented in Table 3. The groups were not significantly different in their ratings of their attention to various course components, including the practice and feedback materials (p = .11); however, the low transactional group rated the relevance of the practice tests and feedback for helping them prepare for course exams and their professional roles significantly higher (p = .03, d = .63) than the low transactional group. Finally, there were no significant differences in the two groups' ratings of their confidence in performing course goals (p = .22) or their personal satisfaction with their efforts during the course (p = .30).

DISCUSSION

Results of this study provide evidence to suggest that transactional distance proximity of practice with feedback does affect students' achievement and aspects of academic motivation. According to Moore's view of distance education, self-directed learners can accommodate greater transactional distance by providing their own structure in a course and creating their own interpersonal dialogue. Many college students fit this description of "field-independent" (Speth, 1991) distance learners; however, others may not, and instructional strategies that decrease transactional distance should improve their learning in distance courses. In this study, upper-division undergraduate students achieved better using the more structured, more interactive practice and feedback format that created less transactional distance during rehearsal.

The mixed results related to students' academic motivation have implications for designers of online instruction. Varying rehearsal formats in terms of their transactional distance did not affect students' attention, confidence, or satisfaction; however, students in the group with lower transactional distance perceived the practice to be more relevant in helping them prepare for exams and their professional roles. This is important since perceptions of relevance in a learning activity can influence students' attention levels. It is also interesting to note that the students in the lower transactional distance group rated their interest and confidence levels as slightly higher than students in the other group, although not statistically so. Replication of this study with a larger group might yield different results in these other factors as well.

A side observation was that students in the more structured, more dialogue group initiated communication with their instructor twice as often as students in the less structure, less dialogue group. The majority of these communications were related to particular questions missed and lower than desired scores received on the quizzes, possibly illustrating either a desire for more in-depth learning or simply a better grade. Regardless, the increased communication indicates increased engagement with the skills included in the rehearsals.

On closer analysis, Moore's transactional distance theory is a higher-level organizing structure for many basic principles found in cognitive psychology and communications. As such, Moore's theory provides a framework and focus for research that falls short of explaining results. Research should continue in strategies for narrowing the transactional distance for students who are less self-directed, but the results of this and future studies should be viewed in light of existing, more basic research in topics such as expectancy, advance organizers, practice and feedback, and time on task. Another interesting line of inquiry might be to investigate whether an interaction exists between the field-dependence/independence trait and transactional distance treatments.

References

Carey, J. O., & Gregory, V. L. (2002). Students' perceptions of academic motivation, interactive participation, and selected pedagogical and structural factors in web-based distance learning. Journal of Education for Library and Information Science, 43(1), 6-15.

Carey L. M., Carey J. O., Dedrick, R., Kushner, S., & Greenbaum, P. (1994). Procedures for designing course evaluation instruments: masked personality model versus transparent achievement model. Journal of Educational and Psychological Measurement, 54, 134-145.

Carnegie Foundation (2005). Carnegie Foundation for the Advancement of Teaching. Retrieved March 16, 2005, from http://www.carnegiefoundation.org/classification.

Chute, A.G., Thompson, M.M., & Hancock, B.W. (1999). The McGraw-Hill handbook of distance learning. New York: McGraw-Hill.

Clark, R. (1983). Reconsidering research on learning from media. Review of Educational Research, 53(4), 445-459.

Dick, W., Carey, L. M., & Carey, J. O. (2005). The systematic design of instruction. (6th Ed.). New York: Allyn & Bacon.

Dihoff, R. E., Brosvic, G. M., Epstein, M. L., & Cook, M. J. (2004). Provision of feedback during preparation for academic testing: Learning is enhanced by immediate but not delayed feedback. The Psychological Record, 54, 207-231.

Gagne, R. M. (1985). Conditions of learning. (4th Ed.) New York: Holt, Rinehart, and Winston.

Keller, J. M. (1987). Development and use of the ARCS model of instructional design. Journal of Instructional Development, 10(3), 2-10.

Moore, M., & Kearsley, G. (1996). Distance education: A systems view. Wadsworth Publishing.

Speth, C. (1991). Important themes and concepts in technology-based distance education: A review of the research literature. Manhattan, Kansas: University of Kansas.

Williams, M. L., Paprock, K., & Covington, B. (1999). Distance learning: The essential guide. Thousand Oaks: Sage Publications.

Willis, B. (1993). Distance education: A practical guide. Englewood Cliffs, NJ: Educational Technology Publications.

Wise, S.L., & DeMars, C.E. (2005). Low examinee effort in low-stakes assessment: Problems and potential solutions. Educational Assessment, 10(1), 1-17.

TARY WALLACE

University of Central Florida, USA

[e-mail]

LYNN GRINNELL

St. Petersburg College, USA

[e-mail]

LOU CAREY

University of South Florida, USA

[e-mail]

JAMES CAREY 

University of South Florida, USA

[e-mail]

Table 2 Mean, Standard Deviation, and Reliability for the Final
Examination: High and Low Transactional Distance Rehearsal Groups

                                     Final Examination
Group                          n   Mean   Std. Deviation

Low Transactional Distance     21  31.15  5.39
High Transactional Distance    24  27.62  4.75
K-[R.sub.20] Test Reliability           .77

Table 3 Means, Standard Deviations, and Reliabilities for Students'
Ratings of Their Attention to Various Instructional Components of the
Course, Relevance of Practice Testing Materials, Confidence in
Performing Course Goals, and Personal/Professional Satisfaction

                        Attention  Relevance  Confidence  Satisfaction
Group               n   Mean  SD   Mean  SD   Mean  SD    Mean  SD

Low Transactional   21  4.18  .76  4.35  .68  3.90  .48   3.78  .63
  Distance
High Transactional  24  3.89  .79  3.92  .80  4.09  .64   3.69  .59
  Distance
Cronbach Alpha             .86        .88        .93         .92
p                          .11        .03*       .22         .30

Note: maximum score = 4 from Likert-type scale. *p<.05.