An empirical investigation of the efficacy of multimedia instruction in counseling skill development. (Counselor Preparation).
The world has experienced a burst of technology that has become increasingly apparent in education and teaching. The use of technology in teaching is improving, developing, and being implemented daily. Educators have realized how powerful and beneficial technology can be in classroom teaching and have integrated this medium into their delivery of instruction (Hayes & Robinson, 2000; Janda, 1992; Jones & Smith, 1992; Poirot, 1992).
In classroom teaching, the use of computers and multimedia instruction, such as distance education (McIsaac & Blocher, 1998; Offir, 2000), the Internet (A. C. Cohen, 1999; Gray, 1998; Wiens & Gunter, 1998), and Web-based instruction (Berge, 1998; Gillani, 1998; Hartley, 1999; Johnson, 1998; Khan, 1998), has increased greatly in recent years. One reason for the wide use of technology in teaching is that the needs of students in today's society are quite different from previous generations' needs. Educators are realizing that traditional methods of teaching (e.g., lecturing) are no longer sufficient to challenge or actively stimulate all students. One way that educators are accommodating the different learning styles of individual students is by using a variety of teaching approaches, including didactic, experiential, computer-assisted, and multimedia instruction (Cairo & Kanner, 1984; Hayes & Robinson, 2000; Hoffman & Waters, 1982; Lawton & Gerschner, 1982; Passey, 2000; Sales, 1999).
Walz (1984) stated that computers have supplemented instruction and have proven to be quite beneficial to students' learning and development processes. Engels, Caulum, and Sampson (1984) pointed out that the potential benefits from computer-based capabilities were too great for counselor educators to ignore. Computer capabilities related to counselor education should be seen as continuous efforts to improve the professional practices, teaching, services, and research of counselor educators. It is unnecessary for counselor educators to be skilled at programming computers, although they must be able to use computers and related tools. As for computer use in counselor education, Engels et al. suggested "that familiarity with the benefits of computers should be a goal of professional growth" (p. 196).
The literature reveals significant interest in the use of multimedia instruction in counselor education (Engels et al., 1984; Hayes & Robinson, 2000), yet there is a limited number of empirical research studies tying the use of multimedia instruction directly to counselor trainees (specifically, Engen, Finken, Luschei, & Kenney, 1994; Hayes & Robinson, 2000). Engen et al. and Hayes and Robinson are the only two studies that could be found. Engen et al. studied the use of interactive laser disc technology. The use of interactive laser disc technology provided realistic situations to which students reacted as they dealt with microcounseling skills. Specifically, four simulated counseling experiences were developed and recorded on videotape and video discs. Engen et al. expressed that "Learning activities, implemented using HyperCard, provided opportunities for students to observe, identify, practice, and evaluate a variety of microcounseling skills" (p. 283). Engels et al. stated that
Critical for counselor educators is the need to help aspiring professionals and those in the field to anticipate relevant areas of ethical concern so that they may appropriately, confidently, and effectively, use the potential strengths computers bring to our profession. (p. 202)
The primary purpose of this study was to investigate the effectiveness of multimedia instruction on counseling skill development. Specifically, we studied the degree to which the use of multimedia adds to the effect of traditional didactic/experiential instruction on counseling skill development by comparing didactic/experiential instruction alone to the same instruction supplemented with multimedia technology. For this study, traditional instruction, as related to counselor education, is defined as verbally presented material (e.g., lecturing) that provides students with the opportunity to participate in role-playing exercises and activities in order to practice developmental counseling skills. Multimedia instruction is defined as the use of computer-based, automated presentation of text material and activities, including the use of quick-time video clips of counseling sessions as well as laser discs of popular movies demonstrating specific behavioral situations and counseling techniques. Our research hypothesis was the following: The combined effect of multimedia and traditional instruction on counseling skill development exceeds the effect of traditional instruction alone to a statistically significant extent.
The participants were graduate students from a major southeastern university enrolled in an introductory course in counseling communication skills. A total of 73 students (20 men and 53 women) participated in the study, with ages ranging from 24 to 47 years. Of the participants in this study, there were 61 European Americans, 9 African Americans, 2 Hispanic Americans, and 1 Native American.
To evaluate and determine the level of counseling skill development of the study participants, a team of six counselor educators was trained to use The Global Scale for Rating Helper Responses (GSRR; Gazda, Asbury, Balzer, Childers, & Walters, 1977). The instrument requires observers to score participants' counseling responses on a 4-point scale indicating that the counseling response was the following: 1 = not helpful: harmful, 2 = not helpful: ineffective, 3 = helpful: facilitative, and 4 = helpful: additive. This scale has been widely used in research studies as a measure of interpersonal communication skills (e.g., May, Powell, Gazda, & Hauser, 1985; Robinson & Wilson, 1980, 1987). The development of GSRR was theoretically founded on the Human Relations Development Model (see Gazda et al., 1999). The content of the scale was closely organized around this model, thereby substantiating its content validity. Evidence concerning the construct validity of the scale was collected by May et al. (1985), who found that patients who were trained with respect to interpersonal communication skills received higher ratings on the scale than did a control group. This demonstrates that the scale was able to detect the growth in interpersonal communication skills facilitated by the treatment, providing some evidence that the scale measures what it purports to measure.
Interpersonal communication skills are a fundamentally important aspect in counseling. In the present study, the GSRR was used to assess how effective students became in using interpersonal communication skills in the context of counseling. The instrument measures the following counseling skills: confrontation, reflection, interpretation, open questions, closed questions, self-disclosure, minimal encourager, empathy, paraphrase, summarization, immediacy, and concreteness. The GSRR was used to calculate each student participant's counseling skill composite. To construct the composite, participant ratings on the scale were equally weighted.
Each participant was enrolled in one of five sections of an introductory counseling communication skills course. Five instructors participated in the study; each instructor taught one section. All classes met once a week for 15 weeks.
The five sections were instructed using one of three treatments (high-tech multimedia, low-tech multimedia, and traditional instruction). Two sections were taught using the high-tech multimedia approach and two using traditional instruction. Only one section was taught using the low-tech multimedia approach.
All three treatments were taught using a didactic/experiential instructional approach. The instructor verbally presented the material and provided students the opportunity to participate in role-play exercises and activities and to practice the presented developmental skills. Furthermore, students participated, when applicable, in the process of note taking, discussing, and questioning while the instructor lectured.
The traditional instruction treatment differed from the other two treatments in that multimedia was not used as a part of instruction. Multimedia instruction included the use of a computer-based, automated presentation of text material and class activities. In addition, the instructor used quick-time video clips of counseling sessions as well as laser discs of popular movies demonstrating specific behavioral situations and counseling techniques. These videos and laser discs were commercially available. This instructional approach was integrated with the didactic/experiential instructional approach.
The only difference between the low-multimedia group and the high-multimedia group was the classroom setting. The high-multimedia group classes were held weekly in a multimedia classroom located in an educational technology center. The low-multimedia group was held weekly in a traditional
classroom setting. The latter group was supplemented with portable computer-assisted instruction (laser disc player, video camera recorder/monitor, liquid crystal display projector, Power Macintosh Computer).
The high-tech multimedia group included 30 students. One section of the high-tech multimedia group consisted of 12 students (Teacher 1) and the other consisted of 18 students (Teacher 2). The low-tech multimedia group contained one section of 14 students (Teacher 3). The traditional instruction group (control group) consisted of two sections. One section had 15 students (Teacher 4) and the other section had 14 students (Teacher 5).
Prior to the study, a team of counselor educators developed a session-by-session curriculum, complete with class objectives and activities to be used by each instructor. In addition, the five instructors completed four pretraining workshops with the first author and the project director. The five instructors met weekly for 2 hours throughout the semester with the first author and the project director to review curriculum materials and instruction methods and to discuss any concerns regarding the classes. These meetings ensured that instruction was consistent and standardized across classes. Furthermore, the first author and several counselor educators made periodic visits to each class in order to monitor the consistency of the instructor's teaching methods.
Only the instructors assigned to the high-tech and low-tech multimedia groups were permitted to interact with the computer and related tools. Students were not allowed to use any of the technological equipment. All five sections shared a common syllabus and required the same textbook. All classes met for a total of 15 weeks. Each class was 2 1/2 hours long. Each participant received graduate credit for the course.
At the beginning of the study, the participants were required to conduct and tape counseling sessions. The pretest video counseling tape provided baseline data of each participant's counseling skills. At the end of the 15-week period, all participants conducted additional counseling sessions that were also taped. The posttest video counseling tape served as the dependent variable in the study.
Six raters were instructed by the first author in standardized scoring procedures to ensure accurate scoring of each counseling response during the videotaped sessions. Each response received a rating. To estimate the level of agreement among raters, a Cronbach's alpha was calculated on the raters' ratings of student counseling skills. Using the Cronbach's alpha as a summary index of interrater consistency is equivalent to obtaining a generalizability coefficient (Crocker & Algina, 1986). The Cronbach's alpha obtained was .91, suggesting that the raters were highly consistent in their application of the rating criteria.
The study was implemented with a pretest-posttest comparison group design. A repeated measures analysis of variance (ANOVA) was conducted to test the effectiveness of multimedia-delivered instruction on counseling skill development. In particular, a statistically significant interaction effect was hypothesized, with students of multimedia-delivered instruction evidencing greater counseling skill development over time than would students of traditionally delivered instruction. To statistically control for the impact of instructor influence, the multimedia condition was nested within "instructor." An alpha level of .05 was used for all the analyses conducted.
Gall, Borg, and Gall (1996) explained that a repeated measures ANOVA is a suitable statistical procedure for determining the statistical significance of pretest-posttest change in experiments. When using this procedure, the interaction between time of measurement and treatment is of particular interest to the analyst. The interaction effect suggests the degree to which the difference between the pretest and posttest means of the experimental group is significantly greater or less than the difference for the control group (see Gall et al., 1996, p. 536).
It is important to highlight that an analysis of covariance (ANCOVA), treating pretest scores as a covariate, was not used in this study. Several authors have cautioned against using ANCOVA as a statistical control procedure in experimental and quasi-experimental designs (Campbell & Erlebacher, 1975; Keppel & Zedeck, 1989; Thompson, 1992).
Descriptive statistics of participants' pre- and posttest video counseling tape composites are displayed in Table 1.
Participants' pretest video counseling tape composite scores ranged from 1.01 t0 3.00, with a mean of 2.13 (SD = .45, n = 73); posttest video counseling tape composites ranged from 2.00 to 3.50, with a mean of 2.92 (SD = .37, n = 73). Descriptive statistics for participants' pre- and posttest video counseling tape scores for the two treatment groups and the one control group are presented in Table 2.
No statistically significant interaction was found using the repeated measures ANOVA, suggesting that multimedia delivered instruction did not influence the rate of development in counseling skills, F(2, 68) = 0.76, p = .472, [[eta].sup.2] = .0008 (see Figure 1). So, only 0.0008% of the variation in the ratings of counseling skill development may be attributable to the impact of multimedia instruction. The size of the effect was much too small to be anything but trivial. Reasons for this are addressed in detail in the Discussion section. It is important to note that a nested model was used to obtain this result to control for the systematic source of variation attributable to teachers nested within the three technology group conditions and the interaction of this nested effect and time (see Table 3 for the within- and between-subject effects for a clearer sense of how the model was specified).
[FIGURE 1 OMITTED]
Conversely, a statistically significant within-subject main effect was found suggesting that all three groups developed better counseling skills as a result of instruction skills, F(1, 68) = 125.472, p = .0001, [[eta].sup.2] = 0.631. This effect size is large, with training accounting for roughly 63% of the variation in acquired counseling skill performances. Such a result strongly suggests that the attainment of statistical significance is not merely due to sample size but due in large part to the impact of training on the growth of the counselors.
A second repeated measures ANOVA was used to investigate the effect teachers had on the counseling skill composite. No statistically significant interaction was attained, suggesting teachers did not differ in their ability to affect the rate at which students developed counseling skills, F(4, 68) = 1.37, p = .255, [[eta].sup.2] = .074. J. Cohen (1977) has indicated that an effect size exceeding .06 in the social sciences may be considered medium in magnitude. Therefore, the fact that teachers influenced the variation in counseling skills by 7.4% is noteworthy enough to raise a question about whether the present study had insufficient power to detect an otherwise moderate effect. If the p value (.255) may be interpreted to say that the estimate in question and associated results fail to have sufficient precision, then an effect size of 7.4% must be viewed skeptically until further replication, particularly with a larger sample, bears this out. The results of the present study may as well be judged to have insufficient power to detect that there is indeed no effect at all. That result aside, a statistically significant main effect for teachers was obtained, F(4, 68) = 4.315, p = .004, [[eta].sup.2] = .202. This effect is largely attributable to which group of students a teacher inherited, given the initial differences evident (see Figure 2).
[FIGURE 2 OMITTED]
The purpose of this study was to investigate the effects of multimedia instruction compared with traditional instructional methods in an introductory class focusing on counseling skill development. Another purpose of the study was to examine if there was a difference between students' skill acquisition within a high-tech multimedia classroom and a low-tech multimedia classroom.
The results of this study showed that there was no statistically significant difference among the levels of students' counseling skill development across the three (high-tech multimedia, low-tech multimedia, and traditional instruction) treatment groups, as measured by the GSRR (Gazda et al., 1977). This suggests that the use of multimedia instruction in the classroom does not improve students' counseling skill development, as measured in this study.
Counseling skill ratings of students' posttest video counseling tapings were higher than the counseling skill ratings of students' pretest video counseling tapings. The data indicate, however, that the observed increase is not due to multimedia instruction. In addition, the results also indicate that there was no difference in counseling skill ratings of students in high-tech as compared with students in low-tech classroom environments. From the results of the study, it appears students can complete a unit of multimedia instruction to attain a level of counseling skill development comparable to the effects of traditional instruction.
Limitations and Future Directions
Given the sophistication of today's technology and the advantages associated with this medium, why was multimedia instruction not more effective in our study? We speculated that one reason the effect apparently did not materialize was due to the instructors' level of comfort working with technology. Instructors who were uncomfortable or intimidated by technology could have impeded students' skill development. Furthermore, the instructors received specific computer training focusing on the development of visual computer presentations; however, they did not receive basic troubleshooting or advanced technological training.
A common limitation associated with the design of studies like this one concerns the fact that it is very difficult to engage a truly satisfactory number of teachers to participate in the study's implementation. Although in this study we nested "counselor training" within "teacher" as a form of statistical control, a better design would be a study in which many more teachers participated, thereby reducing the potential effects of teacher personality and competency.
Finally, future research in examining the effectiveness of multimedia instruction in counselor education should also assess students' counseling skill development in courses using the Internet, Web-based instruction, and distance education.
TABLE 1 Descriptive Statistics for Pretest and Posttest Video Counseling Tape Scores of Students Range Variable M SD Low High Pretest video counseling tape Total sample (N = 73) 2.13 .45 1.01 3.00 High multimedia (n = 30) 2.12 .43 1.10 2.90 Low multimedia (n = 14) 2.19 .44 1.20 3.00 Traditional (n = 29) 2.11 .48 1.20 3.00 Posttest video counseling tape Total sample (N = 73) 2.92 .37 2.00 3.50 High multimedia (n = 30) 2.96 .43 2.00 3.50 Low multimedia (n = 14) 2.80 .42 2.30 3.50 Traditional (n = 29) 2.94 .27 2.40 3.50 TABLE 2 Descriptive Statistics for Pretest and Posttest Video Counseling Tape Scores of Teachers Pretest Posttest Treatment M SD M SD High-tech multimedia Teacher 1 (n = 12) 2.00 .34 2.68 .35 Teacher 2 (n = 18) 2.20 .48 3.15 .37 Low-tech multimedia Teacher 3 (n = 14) 2.15 .44 2.80 .42 Traditional instruction Teacher 4 (n = 15) 2.31 .44 2.98 .30 Teacher 5 (n = 14) 1.90 .43 2.88 .23 Note. n = number of students in class. TABLE 3 Tests of Within-Subject Contrasts and Between-Subject Effects Type III Source SS df MS F Sig. Within-Subject Contrasts Time 20.25 1 20.25 125.47 .000 Time x Technology 0.25 2 0.12 0.76 .472 Time x Teacher (technology) 0.61 2 0.30 1.88 .161 Error (time) 10.98 68 0.16 Between-Subject Effects Intercept 870.24 1 870.24 6041.70 .000 Technology 1.59E-02 2 7.94E-03 0.06 .946 Teacher (technology) 2.44 2 1.22 8.47 .001 Error 9.80 68 0.14 Note. The measure is video counseling tape sources. Time is linear. Transformed variable is average.
Berge, Z. L. (1998). Guiding principles in Web-based instructional design. Educational Media International, 35, 72-76.
Cairo, P. C., & Kanner M. S. (1984). Investigating the effects of computerized approaches to counselor training. Counselor Education and Supervision, 24, 213-220.
Campbell, D. T., & Erlebacher, A. (1975). How regression artifacts in quasi-experimental evaluation can mistakenly make compensatory education look harmful. In M. Guttentag & E. L. Struening (Eds.), Handbook of evaluation research (Vol. 1, pp. 597-617). Beverly Hills, CA: Sage.
Cohen, A. C. (1999). Instructional technology and distance learning through the Internet. Educational Media International, 36, 218-229.
Cohen, J. (1977). Statistical power analysis for the behavioral sciences. New York: Academic Press.
Crocker, L., & Algina, J. (1986). Introduction to classical and modern test theory. New York: Harcourt, Brace, and Jovanovich.
Engels, D. W., Caulum, D., & Sampson, D. E. (1984). Computers in counselor education: An ethical perspective. Counselor Education and Supervision, 24, 193-203.
Engen, H. B., Finken, L. J., Luschei, N. S., & Kenney, D. (1994). Counseling simulations: An interactive videodisc approach. In H. Resnick (Ed.), Electronic tools for social work practice and education (pp. 283-298). Binghamton, New York: The Haworth Press.
Gall, M., Borg, W., & Gall, J. (1996). Educational research (2nd ed.). New York: Longman.
Gazda, G. M., Asbury, F. R., Balzer, F. J., Childers, W. C., Phelps, R. E., & Walters, R. P. (1999). Human relations development. Boston: Allyn & Bacon.
Gazda, G. M., Asbury, F. R., Balzer, F. J., Childers, W. C., & Walters, R. P. (1977). Human relations development (2nd ed.). Boston: Allyn & Bacon.
Gillani, B. (1998). The Web as a delivery medium to enhance instruction. Educational Media International, 35, 197-202.
Gray, S. (1998). Maintaining academic integrity in Web-based instruction. Educational Media International, 35, 186-191.
Hartley, K. W. (1999). Media overload in instructional Web pages and the impact on learning. Educational Media International, 36, 145-150.
Hayes, B. G., & Robinson, E. H. (2000). Assessing counselor education students' attitudes toward computers and multimedia instruction. Journal of Humanistic Counseling, Education and Development, 38, 132-141.
Hoffman, J. L., & Waters, K. (1982). Some effects of student personality on success with computer-assisted instruction. Educational Technology, 22, 20-21.
Janda, K. (1992). Multimedia in political science: Sobering lessons from a teaching experiment. Journal of Educational Multimedia and Hypermedia, 1, 341-354.
Johnson, E. (1998). The World Wide Web, computers and teaching literature. Educational Media International, 35, 203-215.
Jones, L. L., & Smith, S. G. (1992). Can multimedia instruction meet our expectations? Educom Review, 27, 39-43.
Keppel, G., & Zedeck, S. (1989). Data analysis for research designs. New York: Freeman.
Khan, B. H. (1998). Web-based instruction: An introduction. Educational Media International, 35, 63-71.
Lawton, J., & Gerschner, V. T. (1982). A review of the literature on attitudes toward computers and computerized instruction. Journal of Research and Development in Education, 16, 50-55.
May, H. J., Powell, M., Gazda, G. M., & Hauser, G. (1985). Life skills training: Psychoeducational training as mental health treatment. Journal of Clinical Psychology, 41, 359-367.
McIsaac, M. S., & Blocher, J. M. (1998). How research in distance education can affect practice. Educational Media International, 35, 43-47.
Offir, B. (2000). Map for decision making in operating distance learning systems--Research results. Educational Media International, 37, 9-15.
Passey, D. (2000). Developing teaching strategies for distance (out of school) learning in primary and secondary schools. Educational Media International, 37, 45-57.
Poirot, J. L. (1992). Assessing and evaluating student gains. The Computing Teacher, 20, 30-32.
Robinson, E. H., & Wilson, E. S. (1980). Effects of human relations training on indices of skill development and self-concept changes in classroom teachers. Journal for Specialists in Group Work, 5, 163-169.
Robinson, E. H., & Wilson, E. S. (1987). Counselor-led human relations training on a consultation strategy. Elementary School Guidance & Counseling, 22, 124-132.
Sales, G. C. (1999). Creating effective multimedia instruction. Educational Media International, 36, 263-270.
Thompson, B. (1992). Misuse of ANCOVA and related `statistical control' procedures. Reading Psychology: An International Quarterly, 13, 3-18.
Walz, G. (1984). Role of the counselor with computers. Journal of Counseling and Development, 63, 135-138.
Wiens, G., & Gunter, G. A. (1998). Delivering effective instruction via the Web. Educational Media International, 35, 93-99.
B. Grant Hayes, Gordon E. Taub, and Edward H. Robinson III, Department of Child. Family, and Community Sciences, University of Central Florida; Stephen A. Sivo, Department of Educational Research, Technology, and Leadership, University of Central Florida. Correspondence concerning this article should be addressed to B. Grant Hayes, Department of Child, Family, and Community Sciences, University of Central Florida, Orlando, FL 32816-1250 (e-mail: email@example.com).
|Printer friendly Cite/link Email Feedback|
|Author:||Hayes, B. Grant; Taub, Gordon E.; Robinson, Edward H., III; Sivo, Stephen A.|
|Publication:||Counselor Education and Supervision|
|Date:||Mar 1, 2003|
|Previous Article:||Teaching school counselors and school psychologists to work collaboratively. (Counselor Preparation).|
|Next Article:||Influence strategies in the supervisory dyad: an investigation into the effects of gender and age. (Supervision).|