Multicultural group supervision: a comparison of in-person versus Web-based formats.

Despite the need for adequate supervision for school counselor trainees, there are limited supervision opportunities for school counselor trainees (Roberts & Borders, 1994). Group supervision is an effective means of providing clinical supervision to several school counselor trainees concurrently (Borders, 1991; Van Horn & Myrick, 2001). Although utilization of a peer supervision group has been shown to be helpful to both school counselor trainees and professionals (e.g., Agnew, Vaught, Getz, & Fortune, 2000; Kern, 1996), a paucity of empirical studies has documented this efficacy (Starling & Baker, 2000).

For school counselor trainees, peer supervision groups that address multicultural issues may complement the contemporary focus on multicultural issues in many school counselor programs (Constantine, 2001; Hobson & Kanitz, 1996). Attention to multicultural issues in school counselor education programs has been necessitated by the growing diversity of school systems in large urban cities across the United States (Constantine et al., 2001). Over the past few decades, multicultural counseling competence has represented an important goal for many school counselor education programs in order to prepare their trainees to work effectively with diverse cultural populations (Holcomb-McCoy, 2001). Multicultural counseling competence refers to counselors' attitudes/ beliefs, knowledge, and skills in working with individuals from various cultural groups (Sue, Arredondo, & McDavis, 1992). Although previous research has found that school counselors and school counselor trainees with higher levels of formal multicultural counseling education (e.g., coursework and workshops) reported greater amounts of self-perceived multicultural counseling competence (Constantine, 2001), little research has examined the role of multicultural supervision in increasing school counselor trainees' self-reported multicultural counseling competence. There is also an absence of research that has explored the impact of receiving multicultural counseling supervision on aspects of school counselor trainees' demonstrated multicultural counseling competence.

One aspect of demonstrated multicultural counseling competence is the ability to identify and integrate cultural factors into conceptualizations of the etiology and treatment of clients' presenting concerns (i.e., multicultural case conceptualization ability). Conceptualizing clients from a multicultural perspective indicates that school counselor trainees are aware of and can integrate information about various cultural factors into clients' presenting issues and, subsequently, identify an appropriate treatment plan for working with clients based on this information (Constantine & Ladany, 2000). Multicultural case conceptualization ability is comprised of two distinct, but interrelated, dimensions. The first factor or conceptualization is based on school counselor trainees' recognition of factors that may be contributing to the etiology of students' problems. The second conceptualization is based on their thoughts about what might be an effective treatment focus or plan for addressing students' difficulties. These conceptualizations may become increasingly complex as school counselor trainees make associations between and among hypothesized etiologies of presenting concerns and, accordingly, integrate these data into treatment plans (Constantine & Gushue, in press). School counselor trainees' ability to perceive and conceptualize cultural information in a complex and sophisticated manner would have important implications for their ability to work effectively with culturally diverse students. Hence, receiving multicultural supervision presumably would affect these trainees' multicultural case conceptualization ability.

Although the advantages of receiving individual multicultural supervision have been extensively documented (e.g., Constantine, 1997; Duan & Roehlke, 2001), there also may be several advantages to using peer group supervision formats to address multicultural issues with school counselor trainees. For example, peer supervision groups can provide support and encouragement to these trainees and may enhance their clinical skills and promote their personal and professional development (Wilbur, Roberts-Wilbur, Hart, Morris, & Betz, 1994). Moreover, supervision group members may serve as resources to each other by serving as sounding boards, challenging repetitive therapeutic strategies, and supplying meaningful interpretations of therapeutic processes (Agnew et al., 2000). In peer group supervision models, the teaching components of traditional (i.e., individual) supervision are minimized, dependency on "expert" supervisors' opinions and feedback is decreased, and the evaluative component of the traditional supervisory relationship is de-emphasized (Benshoff & Paisley, 1996; Ray & Altekruse, 2000; Wilbur et al.).

In recent years, the World Wide Web has been used to complement face-to-face supervision when trainees and supervisors are at a distance (Casey, Bloom, & Moan, 1994; Janoff & Schoenholtz-Read, 1999; Paisley & McMahon, 2001, Schnieders, n.d.), providing trainees with the support and guidance they need (Van Horn & Myrick, 2001). The most common form of Web-based supervision entails a professional counselor supervising a small group of counselors through the Internet (Van Horn & Myrick), and a weekly correspondence among the group is instituted through emails, bulletin boards, or instant messaging (Myrick & Sabella, 1995). For school counselor trainees, a peer multicultural supervision group on the Web may represent an exciting means of providing supplementary professional support and, subsequently, increasing cross-cultural competence. Because increasing numbers of school counselors and school counselor trainees may be relying on Web-based supervision to address their professional development needs (e.g., Myrick, 1997; Myrick & Sabella), it seems vital to explore the efficacy of this type of supervision format.

Thus, the primary purpose of this study was to compare the effects of in-person and Web-based multicultural supervision peer group formats on supervision satisfaction and multicultural case conceptualization ability, an aspect of multicultural competence. Because of the exploratory nature of our study, we did not delineate specific hypotheses for this investigation.

METHOD

Participants

Forty-five school counselor trainees who were matriculating in a master's degree program located in the northeast region of the United States participated in the study. This sample of students comprised 35 (77.8%) women and 10 (22.2%) men who ranged in age from 22 to 35 years (M = 25.71, SD = 3.01). The racial and ethnic composition of the school counselor trainees was as follows: 28 (62.2%) White Americans, 8 (17.8%) Black Americans, 5 (11.1%) Asian Americans, and 4 (8.9%) Latino(a) Americans. None of the trainees had any previous counseling experience, and all of them had taken at least one course related to multicultural or cross-cultural counseling issues.

Procedure

School counselor trainees were asked to participate in a semester-long research project focusing on examining two types of peer group supervision formats. They were aware that involvement in the project meant being randomly assigned to participate in either an in-person peer supervision group or a Web-based peer supervision group, each of which focused on multicultural supervision issues. They were also told that all of their supervision sessions would be monitored by one of the researchers either by audiotape (only for the in-person supervision groups) or by reading transcripts of the text generated by the Web-based supervision groups. All of the participating school counselor trainees reported owning a personal computer and familiarity with using a Web browser.

The Web-based and in-person multicultural supervision peer groups met for one hour weekly for 13 weeks as an adjunct to an hour of weekly individual supervision by certified school counselors in their practicum settings. Twenty-one of the school counselor trainees were chosen to participate in the Web-based multicultural supervision peer groups, and 24 trainees were selected for the in-person multicultural supervision peer groups. For the Web-based peer supervision groups, the 21 trainees were divided into three groups comprising 7 students each. The in-person peer supervision groups were divided into three groups consisting of 8 individuals. School counselor trainees participating in the Web-based multicultural supervision peer groups were given instructions on how to access a Web page developed for the purposes of the investigation. They were given a unique user name and password allowing them access to a synchronous Web-based Internet relay chat room to conduct real-time dialogues.

Two supervisors with Ph.D. degrees in counseling psychology and with school counselor certification moderated the Web-based and in-person multicultural supervision peer groups. One of the supervisors was responsible for facilitating all of the peer multicultural supervision Web-based groups and the other supervisor led all of the in-person supervision groups. Both of these supervisors had received an extensive amount of multicultural counseling education and supervision in their graduate programs. For the purposes of the present study, these supervisors received 4 hours of instruction in the use of (a) Constantine's (1997) multicultural supervision framework and (b) strategies for interacting effectively with participants in their respective multicultural group supervision formats. Constantine's conceptual framework, which can also be slightly modified for use in the context of counseling relationships, involves a set of semi-structured questions that aid participants in identifying and understanding how their cultural group identities (e.g., race, ethnicity, gender, social class) influence their multicultural counseling awareness, knowledge, and skills. Specifically, questions in the framework encourage counselors to discuss (a) how their worldviews and value systems may affect their working relationships with culturally diverse students; (b) their knowledge, skills, and challenges in working with culturally diverse students; and (c) ways in which they could improve their skills in working with culturally diverse students.

To evaluate delivery and integrity of the two types of supervision formats using Constantine's (1997) multicultural supervision framework, two procedures were implemented. First, the in-person group supervision sessions were audiotaped each week, and one of the researchers provided feedback to the facilitator based on the content of the tapes. Second, transcripts from the Web-based supervision groups were generated after the termination of the supervision session, and one of the researchers provided feedback to the facilitator based on the transcripts. Each of the supervisors met with one of the researchers individually for two hours per week to ensure treatment integrity.

Prior to participation in the Web-based and in-person multicultural supervision peer groups, the trainees were asked to complete pretest measures in person that included a brief demographic questionnaire and a multicultural case conceptualization ability exercise. The posttest measures were also completed in person and included the same multicultural case conceptualization ability exercise used at pretest and the Supervisee Satisfaction Questionnaire (Ladany, Hill, Corbett, & Nutt, 1996). A code number was used to match pretest and posttest data for the participating trainees, and all interested trainees signed a consent form. The school counselor trainees who participated in the study were provided information about Constantine's (1997) multicultural framework in their respective peer supervision formats and were encouraged to use this framework within their peer supervision groups to help them discuss cases involving the students with whom they worked.

After the study was completed, all of the school counselor trainees were debriefed individually by research assistants and were provided with an opportunity to ask questions or to learn additional information about the investigation. No monetary incentives were provided for their participation.

Instruments

Demographic questionnaire. Participants were asked to indicate their race or ethnicity, sex, age, and the number of formal courses they had taken previously related to cross-cultural or multicultural issues.

Multicultural case conceptualization exercise. For the multicultural case conceptualization ability measure, participants were asked to imagine that they were the school counselor for a student whom they were about to meet. They were provided with a vignette in which the student, who was described as a 10-year-old African American male, was matriculating in a predominantly White suburban school setting and was being referred to counseling because of difficulties in adjusting to a new school, social isolation from peers, lack of motivation for school work, and mildly depressed affect. Thus, there were several potential cultural and mental health issues that could be pertinent to the formulation of a treatment plan. After reading the vignette at both pretest and posttest, the participants were asked to write a conceptualization of at least three sentences describing what they believed to be the etiology of the student's difficulties, and to write a conceptualization of at least three sentences delineating what they believed to be effective treatment strategies or foci for addressing the student's difficulties.

To assess multicultural case conceptualization ability, a coding system was developed by which raters indicated the extent to which participants had integrated salient racial or ethnic issues into the etiology and treatment conceptualizations of the student's presenting concerns. The coding system was based on similar coding systems for assessing integrative complexity (Tetlock & Suedfeld, 1988) and has been used and validated in previous investigations (e.g., Constantine & Gushue, in press; Constantine & Ladany, 2000; Ladany, Inman, Constantine, & Hofheinz, 1997). In this study, multicultural case conceptualization ability was assessed by examining two interrelated cognitive processes. The first process, differentiation, is defined as school counselor trainees' ability to offer alternative interpretations of a student's presenting problems and the nature of the treatment that could be provided. The greater the number of options presented in relation to a student's problems, the greater the degree of differentiation. The second process, integration, is characterized by school counselor trainees' ability to develop associations between and among differentiated interpretations. In previous studies, integrative complexity coding systems have been validated (e.g., Streufert & Streufert, 1978; Tetlock, 1986; Tetlock, Hannum, & Micheletti, 1984) and have been noted to have high interrater agreement (e.g., .87; Tetlock & Kim, 1987).

The raters were two advanced doctoral students in counseling psychology who were trained for 8 hours in the coding of multicultural case conceptualization ability. These raters were unaware of the study's purposes and coded all of the open-ended etiology and treatment responses for multicultural case conceptualization ability. Multicultural conceptualization scores ranged from 0 to 5: zero = no differentiation, no integration (i.e., no indication of racial or ethnic issues in conceptualizing the student's problems) 3 = moderate differentiation, low integration (i.e., two or more references to racial or ethnic issues in the conceptualization of the student's problems, with one connection made between the two or more differentiated concepts); and 5 = high differentiation, high integration (i.e., three or more references to racial or ethnic issues in conceptualizing the student's problems, with three or more connections made between differentiated concepts). Final interrater agreements for the etiology and treatment ratings were .90 and .92, respectively.

Supervisee Satisfaction Questionnaire (SSQ). The SSQ (Ladany et al., 1996) is an 8-item scale that assesses supervisees' satisfaction with the supervision they receive. Scores for each item range from 1 to 4, with higher scores reflecting greater satisfaction (total scores range from 8 to 32). A sample item from the SSQ includes, "To what extent has this supervision fit your needs?" The SSQ was developed from the Client Satisfaction Questionnaire (Larsen, Attkisson, Hargreaves, & Nguyen, 1979), and previous supervision research has reported the internal consistency of the SSQ to range from .96 to .97 (Ladany et al., 1996; Ladany, Lehrman-Waterman, Molinaro, & Wolgast, 1999). In our study, the Cronbach's alpha for the SSQ was .95.

RESULTS

For school counselor trainees participating in the Web-based multicultural supervision peer groups, the mean pretest and posttest scores for the multicultural case conceptualization etiology index were 2.00 (SD = 1.00) and 2.57 (SD = 1.03), respectively, and the mean pretest and posttest scores for the multicultural case conceptualization treatment index were 1.90 (SD = .94) and 2.19 (SD = 1.32), respectively. For school counselor trainees in the in-person multicultural supervision peer groups, the mean pretest and posttest for the multicultural case conceptualization etiology index were 2.25 (SD = 1.26) and 3.29 (SD = 1.08), respectively, and the mean pretest and posttest scores for the multicultural case conceptualization treatment index were 1.79 (SD = 1.14) and 2.75 (SD = 1.22), respectively. The etiology and treatment scores for the entire sample were positively correlated at both pretest (r = .61, p < .001) and posttest (r = .72, p < .001), No statistically significant differences were found at pretest between the Web-based and in-person supervision peer groups with regard to school counselor trainees' multicultural case conceptualization etiology and treatment scores, F(2, 42) = .78, p > .05. However, school counselor trainees reported significant differences in satisfaction with supervision based on supervision peer group formats, t(43) = 4.56, p < .001. Specifically, trainees participating in the in-person multicultural supervision peer group formats reported significantly higher supervision satisfaction ratings (M = 28.08, SD = 3.80) than did their counterparts in the Web-based conditions (M = 22.33, SD = 4.66).

A multivariate analysis of covariance (MANCOVA) was conducted to examine posttest differences between the Web-based and in-person multicultural supervision peer groups on the multicultural case conceptualization variables (i.e., etiology and treatment), after controlling for school counselor trainees' pretest scores on these variables. The overall MANCOVA was significant, F(2, 40) = 4.65, p < .05. Follow-up univariate analyses of covariance revealed that school counselor trainees assigned to the in-person multicultural supervision peer groups obtained higher multicultural case conceptualization etiology scores, F(1, 41) = 8.74, p < .01, and treatment scores, F(1,41) = 4.80, p < .05, than did their peers in the Web-based multicultural supervision peer groups.

DISCUSSION

Consistent with previous research on peer group supervision (Crutchfield & Borders, 1997; Starling & Baker, 2000), the findings in this study offer support for the use of peer group multicultural supervision formats to increase school counselor trainees' abilities to conceptualize cases from a multicultural perspective. In fact, trainees' multicultural case conceptualization ability increased with both in-person and Web-based peer group supervision formats. However, school counselor trainees who participated in-person peer group multicultural supervision demonstrated greater multicultural case conceptualization ability than did trainees who participated in Web-based peer group multicultural supervision. Greater multicultural case conceptualization ability among the in-person group supervision members may be attributed to factors that are vital to effective supervision, but that may be limited in a Web-based format. For example, the lack of perceptual relationship cues in online supervision may render supervisors (and trainees) unable to accurately perceive nonverbal and verbal information (Kanz, 2001) that is critical to assisting supervisees in developing multicultural counseling competence (Constantine, 1997). Further, supervisees may omit critical information that might otherwise be explored in in-person supervision (Janoff & Schoenholtz-Read, 1999). Ladany et al. (1996) found that one third of the psychotherapy supervisees in their study reported that they did not disclose to their supervisors negative feelings about or reactions to their clients. In an effort not to appear prejudiced or bigoted toward their clients or students, some school counselor trainees may be reluctant to report any negative feelings or attitudes about students of color. The exploration of bias is an important aspect of multicultural supervision (Ancis & Ladany, 2001), and facilitating emotionally laden discussions about supervisees' cultural identities, attitudes, values, behaviors, and biases may be more difficult to do in a Web-based peer supervision format than in an in-person format.

Lack of perceptual relationship cues may also have attributed to greater satisfaction with in-person peer group supervision than with Web-based peer group supervision. According to Myrick and Sabella (1995), school counselor trainees and practicing counselors reported more advantages than disadvantages to Web-based supervision, including more accessible assistance and encouragement, interpersonal closeness and openness, the ability to read and review at one's convenience, and spirited conversations. In our study, it is possible that differences in satisfaction between the two supervision formats may also be accounted for by the participants' perceptions of intimacy. An appropriate level of professional intimacy is necessary for effective supervision. Supervisees must self-disclose in a trusting relationship with their supervisor and, in the case of group supervision, their peers. Robson and Robson (1998) suggested that although computer technology can be used for intimate communication, there are dangers that may increase personal barriers to intimacy (e.g., impersonation and impersonalization). Lack of nonverbal communication and paralinguistics can further impede interpersonal intimacy (Robson & Robson). In fact, social isolation was an ongoing theme for the supervisees in Christie's (n.d.) study. Therefore, given the direct access to personal contact, nonverbal communication, and paralinguistics, it is possible our school counselor trainees in the in-person peer group multicultural supervision format experienced their interactions as more intimate than did trainees in the Web-based format.

Implications for School Counselor Education and Practice

The results of our study indicated that in-person peer group multicultural supervision was more effective in developing abilities in multicultural case conceptualization in school counselor trainees than was Web-based supervision. Thus, school counselor education programs should attempt to provide face-to-face peer group multicultural supervision to their trainees when possible. In cases where ongoing in-person peer group multicultural supervision is impossible or so impractical that it may be deemed less effective or problematic (e.g., because of extensive commuting time), Web-based peer group supervision may be the best available option.

This latter issue may be particularly true for practicing school counselors who often do not participate in ongoing clinical supervision despite its importance and advantages (Crutchfield & Borders, 1997; Magnuson, Norem, & Bradley, 2001; Paisley & McMahon, 2001). There is some evidence that group supervision fosters confidence and professionalism in school counselors (Agnew et al., 2000). However, some of the difficulties associated with obtaining supervision have included limited funding, increasing concerns about direct service time, and lack of awareness of the benefits of supervision (Crutchfield & Borders; Paisley & McMahon).

Further, as the cultural diversity in U.S. schools increases (Lee, 1995), school counselors will need to integrate multicultural supervision as an important professional development tool for increasing their multicultural counseling competence (Hobson & Kranitz, 1996). In a position statement, the American School Counselor Association (1999) encourages school counselors to take action to ensure that appropriate services support the maximum development of culturally diverse students. Designing and implementing culturally responsive school programs becomes a "cutting edge" endeavor for the practicing school counselor (Lee, 2001). Peer group multicultural supervision (particularly using a Web-based format) may be the only opportunity for practicing school counselors to develop the necessary skills for addressing the developmental needs of all students.

As our study suggests, however, because in-person supervision may be more effective than Web-based supervision for many people, some combination of the two might be necessary for those for whom regular face-to-face supervision is difficult to execute. It is also feasible to consider that the effectiveness of Web-based supervision may be improved through the use of more interactive forms of computer technology (e.g., videoconferencing; Kanz, 2001; Sampson, Kolodinsky, & Greeno, 1997) or the integration of structured group supervision models (e.g., Benshoff & Paisley, 1996; Wilbur et al., 1994). Myrick and Sabella (1995) reported that participants engaged in Web-based supervision found structured group activities helpful.

When online supervision is used, several ethical issues need to be considered. For example, some authors (e.g., Langford, 1996; Sampson et al., 1997) have been concerned about inappropriate individuals accessing confidential information, although efforts are continually being made to improve the confidentiality on online communications (e.g., encryption; Kanz, 2001). Additional ethical concerns related to online supervision for school counselor trainees include limited training in using computers (Owen & Weikel, 1999; Van Horn & Myrick, 2001), resistance of some school counselors to computer technology (Myrick & Sabella, 1995), and the potentially marked dissimilarities of this method to in-person supervision. With regard to the latter point, although there are both positive and negative issues associated with online supervision relationships (Janoff & Schoenholtz-Read, 1999), vital relationship characteristics (e.g., transference, parallel processes) that may be present in face-to-face supervision may not exist in online supervision (Kanz). These potential concerns notwithstanding, the effectiveness of Web-based supervision (as compared to in-person supervision) for pre-service and practicing school counselors should continue to be assessed because of its increasing popularity as a viable means of providing professional support and development (Owen & Weikel; Van Horn & Myrick).

Limitations of the Study

There are several potential limitations to our study. First, the sample size was small, possibly limiting the variability in responses to the measures. Further, the participants were enrolled in a single graduate program in the northeast region of the United States. In addition, it is possible that participants assigned to the Web-based supervision format may have needed a refresher course or additional technical supports that might have enabled them to more fully benefit from the positive experiences that participants in other studies have reported regarding the use of Web-based supervision (e.g., Myrick & Sabella, 1995). Lastly, despite our efforts to ensure treatment integrity, there may have been some supervisor-treatment confound effects.

Suggestions for Future Research

Additional research should be conducted with students from other geographical regions. Also, future research should attend to the effectiveness of more interactive forms of computer technology (e.g., desktop videoconferencing or videoteleconferencing) or the use of Web-based supervision as a supplement to in-person supervision (Gammon, Sorlie, Bergvik, & Sorensen-Hoifodt, 1998; Janoff & Schoenholtz-Read, 1999). Further, there is a need to more closely examine the factors that facilitate both satisfaction and dissatisfaction with Web-based peer group multicultural supervision. As technology continues to advance and improve and as the ethical issues associated with Web-based supervision are more adequately addressed, it is conceivable that Web-based supervision will become more frequently used in school counselor education programs. In the meantime, our study's findings suggest that when compared to Web-based supervision, in-person peer group multicultural supervision offers a more effective and satisfying means of developing multicultural case conceptualization ability in school counselor trainees. It may be that in situations where no in-person peer group multicultural supervision is available, Web-based peer group multicultural supervision is better than no supervision at all. In such cases, it would be critical that supervisors and supervisees address the practical limitations (e.g., perceptual cues, financial resources) and ethical and legal considerations (e.g., confidentiality, licensure regulations) of using electronic communication as a means of conducting clinical supervision (Janoff & Schoenholtz-Read; Kanz, 2001).

References

Agnew, T., Vaught, C. C., Getz, H. G., & Fortune, J. (2000). Peer group clinical supervision program fosters confidence and professionalism. Professional School Counseling, 4, 6-12.

American School Counselor Association. (1999). Position statement: Cross/multicultural counseling. Alexandria, VA: Author.

Ancis, J. R., & Ladany, N. (2001). A multicultural framework for counselor supervision. In L. J. Bradley & N. Ladany (Eds.), Counselor supervision: Principles, process, and practice (3rd ed., pp. 63-90). Philadelphia, PA: Brunner-Routledge.

Benshoff, J. M., & Paisley, P. O. (1996). The structured peer consultation model for school counselors. Journal of Counseling and Development, 74, 314-318.

Borders, L D. (1991). A systematic approach to peer group supervision. Journal of Counseling and Development, 69, 248-252.

Casey, J. A., Bloom, J. W., & Moan, E. R. (1994). Use of technology in counselor supervision. Greensboro, NC: ERIC Clearinghouse on Counseling and Student Services. (ERIC Documentation Reproduction Service No. ED372357)

Christie, B. S. (n.d.). Counseling supervision experiences of distance clinical supervision. Retrieved March 12, 2002, from http://cybercounsel.uncg.edu/book/manuscripts/tenets. htm

Constantine, M. G. (1997). Facilitating multicultural competency in counseling supervision: Operationalizing a practical framework. In D. B. Pope-Davis & H. L. K. Coleman (Eds.), Multicultural counseling competencies: Assessment, education and training, and supervision (pp. 310-324). Thousand Oaks, CA: Sage.

Constantine, M. G. (2001). Theoretical orientation, empathy, and multicultural counseling competence in school counselor trainees. Professional School Counseling, 4, 342-348.

Constantine, M. G., Arorash, T. J., Barakett, M. D., Blackmon, S. M., Donnelly, P. C., & Edles, P. A. (2001). School counselors' universal-diverse orientation and aspects of their multicultural counseling competence. Professional School Counseling, 5, 13-18.

Constantine, M. G., & Gushue, G. V. (in press). School counselors' ethnic tolerance attitudes and racism attitudes as predictors of their multicultural case conceptualization of an immigrant student. Journal of Counseling and Development.

Constantine, M. G., & Ladany, N. (2000). Self-report multicultural counseling competence scales: Their relation to social desirability attitudes and multicultural case conceptualization ability. Journal of Counseling Psychology, 47, 155-164.

Crutchfield, L. B., & Borders, L. D. (1997). Impact of two clinical peer supervision models on practicing school counselors. Journal of Counseling and Development, 75, 219-230.

Duan, C., & Roehlke, H. (2001). A descriptive "snapshot" of cross-racial supervision in university counseling center internships. Journal of Multicultural Counseling and Development, 29, 131-146.

Gammon, D., Sorlie, T., Bergvik, S., & Sorensen-Hoifodt, T. (1998). Psychotherapy supervision conducted via videoconferencing: A qualitative study of users' experiences. Nordic Journal of Psychiatry, 52, 411-442.

Hobson, S. M., & Kanitz, H. M. (1996). Multicultural counseling: An ethical issue for school counselors. The School Counselor, 43, 245-255. Holcomb-McCoy, C. C. (2001). Exploring the self-perceived multicultural counseling competence of elementary school counselors. Professional School Counseling, 4,195-201.

Janoff, D. S., & Schoenholtz-Read, J. (1999). Group supervision meets technology: A model for computer-mediated group training at a distance. International Journal of Group Psychotherapy, 49, 255-272.

Kanz, J. E. (2001). Clinical-supervision.com: Issues in the provision of online supervision. Professional Psychology: Research and Practice, 32, 415-420.

Kern, C. (1996). Counseling supervision for school counselors' professional growth. TCA Journal, 24(2), 1-8.

Ladany, N., Hill, C. E., Corbett, M. M., & Nutt, E. A. (1996). Nature, extent, and importance of what psychotherapy trainees do not disclose to their supervisors. Journal of Counseling Psychology, 43,10-24.

Ladany, N., Inman, A. G., Constantine, M. G., & Hofheinz, E.W. (1997). Supervisee multicultural case conceptualization ability and self-reported multicultural competence as functions of supervisee racial identity and supervisor focus. Journal of Counseling Psychology, 44, 284-293.

Ladany, N., Lehrman-Waterman, D., Molinaro, M., & Wolgast, B. (1999). Psychotherapy supervisor ethical practices: Adherence to guidelines, the supervisory working alliance, and supervisee satisfaction. The Counseling Psychologist, 27, 443-475.

Langford, D. (1996). Ethics and the Internet: Appropriate behavior in electronic communication. Ethics and Behavior, 6, 91-106.

Larsen, D. L., Attkisson, C. C., Hargreaves, W. A., & Nguyen, T. D. (1979). Assessment of client/patient satisfaction: Development of a general scale. Evaluation and Program Planning, 2,197-207.

Lee, C. C. (Ed.). 1995). Counseling for diversity: A guide for school counselors and related professionals. Boston: Allyn & Bacon.

Lee, C. C. (2001). Culturally responsive school counselors and programs: Addressing the needs of all students. Professional School Counseling, 4, 257-261.

Magnuson, S., Norem, K., & Bradley, L. J. (2001). Supervising school counselors. In L. J. Bradley & N. Ladany (Eds.), Counselor supervision: Principles, process, and practice (pp. 207-221). Philadelphia, PA: Brunner-Routledge.

Myrick, R. D. (1997). Traveling together on the road ahead. Professional School Counseling, 1(1), 4-8.

Myrick, R. D., & Sabella, R. A. (1995). Cyberspace: New place for counselor supervision. Elementary School Guidance and Counseling, 30, 35-44.

Owen, D., Jr., & Weikel, W. (1999). Computer utilization by school counselors. Professional School Counseling, 2, 179-182.

Paisley, P. O., & McMahon, G. (2001). School counseling for the 21st century: Challenges and opportunities. Professional School Counseling, 5, 106-115.

Ray, D., & Altekruse, M. (2000). Effectiveness of group supervision versus combined group and individual supervision. Counselor Education and Supervision, 40, 19-30.

Roberts, E. B., & Borders, L. D. (1994). Supervision of school counselors: Administrative, program, and counseling. The School Counselor, 41, 149-157.

Robson, D., & Robson, M. (1998). Intimacy and computer communication. British Journal of Guidance and Counselling, 26, 33-41.

Sampson, J. P., Jr., Kolodinsky, R.W., & Greeno, B. P. (1997). Counseling on the information highway: Future possibilities and potential problems. Journal of Counseling and Development, 75, 203-212.

Schnieders, H. L. (n.d.). From a bug in the ear to a byte in the eye: Implications for Internet-delivered, live counselor supervision. Retrieved March 12, 2002, from http://cybercounsel.uncg.edu/book/manuscripts/Schnieders.htm

Starling, P. V., & Baker, S. B. (2000). Structured peer group practicum supervision: Supervisees' perceptions of supervision theory. Counselor Education and Supervision, 39, 163-176.

Streufert, S., & Streufert, S. (1978). Behavior in the complex environment. Washington, DC: Winston.

Sue, D.W., Arredondo, P., & McDavis, R. J. (1992). Multicultural counseling competencies and standards: A call to the profession. Journal of Multicultural Counseling and Development, 20, 64-68.

Tetlock, P. E. (1986). A value pluralism model of ideological reasoning. Journal of Personality and Social Psychology, 50, 819-827.

Tetlock, P. E., Hannum, K., & Micheletti, P. (1984). Stability and change in senatorial debate: Testing the cognitive versus rhetorical style hypotheses. Journal of Personality and Social Psychology, 46, 979-990.

Tetlock, P. E., & Kim, J. I. (1987). Accountability and judgment processes in a personality prediction task. Journal of Personality and Social Psychology, 52, 700-709.

Tetlock, P. E., & Suedfeld, P. (1988). Integrative complexity coding of verbal behavior. In C. Antaki (Ed.), Analysing everyday explanation (pp. 43-59). London: Sage.

Van Horn, S. M., & Myrick, R. D. (2001). Computer technology and the 21st century school counselor. Professional School Counseling, 5, 124-130.

Wilbur, M. P., Roberts-Wilbur, J., Hart, G. M., Morris, J. R., & Betz, R. L. (1994). Structured group supervision (SGS): A pilot study. Counselor Education and Supervision, 33, 262-279.

Kathy A. Gainor, Ph.D., is an assistant professor in the Department of Counseling, Human Development and Educational Leadership at Montclair State University, Upper Montclair, NJ. E-mail: gainork@mail. montclair.edu. Madonna G. Constantine, Ph.D., is an associate professor in the Department of Psychology at The Ohio State University, Columbus.