Evaluation of a school district's secondary counseling program.

In this age of accountability, all facets of public education are undergoing scrutiny. High-stakes testing, a perceived increase in violence in schools, and tighter budgets combine to heighten both awareness and anxiety of parents, legislators, and other school patrons. School counseling programs, like every aspect of the public schools, are receiving close attention from the public and calls for accountability and evaluation of these programs can be expected to increase.

Evaluation of school counseling programs is not an advent of the millennium. School counselors have long been exhorted to evaluate their programs and to account for their time and effectiveness (Perusse, Goodnough, & Noel, 2001). In his influential book Guidance and Counseling in the Schools, Herr (1979) asserted that guidance programs "... should contain goals, objectives, activities, and student outcomes" (p. 141) and "... some evaluation should be included each year with periodic evaluation of the total program" (p. 142). Lombana (1985) noted that individual school counselors may be prevented from developing evaluations by time constraints, lack of knowledge of evaluation procedures, and anxiety regarding the possible outcomes. An externally based evaluation is of use to time-constrained practitioners whose expertise may not be evaluation. However, Hogan (1998) advocated involvement of the counselors in evaluation of their own programs as a way of keeping the counselor involved in students' daily lives. The American School Counselor Association (ASCA, 1998) addresses the issue of evaluation in Ethical Standard D1.e by advocating that school counselors "... assist in the development of ... a systematic evaluation process for comprehensive school counseling programs, services, and personnel. The counselor is guided by the findings of the evaluation data in planning programs and services." However, counselors may lack the time, the education, or the credibility to conduct their own program evaluations, especially if the competence or expertise of the counselors themselves is an issue in the evaluation.

The evaluation reported here used professionals who were not employees of the school system commissioning the evaluation. The use of evaluators from outside the system to review school counseling programs is an established way of conducting an evaluation (Schmidt, 1995), and precludes the involvement of teachers as evaluators, which is sometimes the chosen method, but outside the teachers' role (Studer & Sommers, 2001). Similarly, principals, although frequently the primary or only evaluators of counselors and counseling programs, usually lack background in counseling practice and are therefore unable to thoroughly, objectively, and professionally evaluate the counselors' goals, adherence to ethical guidelines, and daily activities, and often interpret the counselors' role differently than do the counselors (Fitch, Newby, Ballestero, & Marshall, 2001).

This article details the experience of three university-based external evaluators of a secondary school counseling program and suggests some methods that worked for this evaluation. These suggestions may guide other evaluation efforts, whether the evaluators be internal supervisors or administrators of a school system or external consultants such as counselor educators. The article focuses on the process, rather than the findings, of the evaluation, that others may find usable in their work settings or in their planning of external evaluations.

THE EVALUATION

The superintendent of a mid-sized (about 12,000 students) school district approached the first author, a counselor educator, about evaluating the secondary school counseling program in the district. Constant and intense criticism from parents to school board members prompted the superintendent's need for data and recommendations for changes in the school counseling program to answer these critics and, more importantly, to improve service to students and their parents. He determined that outside evaluators were needed both for their skills and for their credibility in making recommendations.

In response to the superintendent's request, the counselor educator, assisted by an educational research professor, prepared a brief proposal outlining suggested methodology, timeline for the work, form of the final report, and budget. Following acceptance of the proposal by the school board and superintendent, evaluation instruments were devised. Doctoral students, who had prior experience and background in conducting interviews and focus groups and in analyzing data, assisted in collecting, aggregating, and analyzing data. The counselor educator prepared and presented the final report.

The Evaluators

The evaluators for this program included a counselor educator with more than 20 years of experience as a school counselor and central office administrator, who served as primary evaluator, and two doctoral students, one in counselor education with school counseling experience and one in human development with extensive program management and evaluation experience. They were assisted in the design of the evaluation by an educational research professor with expertise in program evaluation in public schools. This partnership of evaluators followed the recommendation that interdisciplinary collaboration is considered an important component of the overall evaluation process (Hayes, Paisley, Phelps, Pearson, & Salter, 1997). None of the evaluators had any recent significant history with the school system, although the counselor educator knew the superintendent and was acquainted with some of the counselors and administrators through past professional association work and consulting with the school district.

Background

The county, congruent with the school district, is a historically stable political entity which is currently experiencing significant growth. The superintendent was relatively new to the district, following a long-tenured superintendent, and there had been many changes in administrators and counselors over the past few years. Several new schools opened recently. Many of the newcomers are affluent parents who sometimes have different expectations than those traditionally found in the formerly rural setting. Among other demands, the children from affluent families are applying to many more colleges and seeking more financial aid, thus requiring more and different service from school counselors than in previous years.

Planning the Evaluation

The evaluators realized they were facing a difficult task for several reasons. The counselors did not want their program to be evaluated, and felt frustration and anger in being singled out for this unwanted attention. Because there was considerable criticism of the counseling program, the evaluators knew many people would be scrutinizing the report, which needed to be as unassailable as possible in methodology and objectivity. Since the school district wanted the report as quickly as possible, the evaluators needed to be realistic in timelines and careful to stay separate from the politics of the school district. In consultation with professors of educational research and educational administration, the primary evaluator developed some principles to guide the evaluation to ensure as much credibility and validity as possible for the final report. The following principles were followed and may be of use to other professionals, either external or internal to a school system, planning an evaluation of a school counseling program:

1. The evaluation would take time and involve many constituents and data sources, not be "quick and dirty." The superintendent and district staff members were persuaded that accurate data could be gained only over a period of months, not a few days. They also concluded that lasting change was more likely if everyone, particularly the counselors, agreed on the need for the evaluation and had sufficient opportunity to participate. Interviews and focus groups were selected as the primary means of gathering data. Interviews gain much detail from participants, but are very time-consuming and not suitable for all data gathering from constituent groups. Thus, the evaluators decided that focus groups would be an appropriate data-collecting method when individual interviews were prohibited by use of time. The use of focus groups is one way of directly involving participating groups--in this case faculty, students, and parents in each high school--in the process (Mueller & Krueger, 1986) and is also a recommended method for the evaluation of programs that directly affect students (Kramer, 1992). It is also described as a useful process by which a facilitator can assure input and feedback from a variety of perspectives, especially when group participants vary in ethnicity, gender, or in other ways that might influence their thinking (Morrow, Burris-Kitchen, & Der-Karabetian, 2000).

2. Sufficient credibility had to be established by the evaluators so counselors, parents, students, and administrators would willingly share accurate information. This was accomplished in part by a serendipitous happening. The primary evaluator was invited to present a workshop to all counselors in the school district in early August, by a new central office staff member who had not yet been informed about the planned evaluation. She accepted the invitation, on condition that she have an initial, extended meeting with secondary counselors following the workshop, where the evaluation plan would be explained by a district administrator and then by the evaluator. At this meeting, she asked the counselors which areas they wished to be able to address and an initial list of problem areas was developed from the counselors. After the counselors became reassured that they would be involved, they were very cooperative and eager to assist with the logistics of the evaluation. Fairly quickly, the mood seemed to change from having something done to them, to an attitude of participation.

3. Counselors' confidentiality had to be maintained. Because there were several counselors at each of the three high schools, group data could be reported for each school without individual responses, though quoted, being attributable to an specific individual. This principle worked in every instance except for one counselor's daily log, when she in effect revealed her identity by the way she chose to log her activity. Principals' anonymity could not be promised, since there was only one principal at each high school and, in some sections of the final report, some data were organized by school.

4. The process would be designed in a way that minimized intrusion on everyone's workdays, especially counselors' workdays. From early conversations and the evaluators' knowledge of public school demands, it was clear that lack of time to accomplish all tasks was a major problem. An intrusive, time-consuming evaluation process would produce further frustration and even contribute to the problem. Thus, the evaluators endeavored to encumber each individual's time as little as possible. Again, the focus group proved valuable to limit expenditure of individual time, while allowing individual expression of needs and opinions in an open, supportive forum.

5. The findings would be anchored to something other than just evaluators' opinions about what should be. Hopefully, both counselors and school administration would continue their own planning using these resources after the evaluators had completed their work. Credibility of the final report would be enhanced by basing recommendations, such as counselor/student ratio and program activities, on a recognized set of standards or recommendations. The National Standards for School Counseling Programs (ASCA, 1999a; Campbell & Dahir, 1997) and ASCA's (1999b) Role of the Professional School Counselor proved ideal for this purpose and were included as appendices in the final report.

6. The report would be as jargon-free and clear as possible since it was ultimately to be presented to a lay group, the school board.

Devising Evaluation Methodology

The evaluators determined that they would use seven sources of data as follows: (a) existing documents such as guidance handbooks, minutes of counselors' monthly meetings, annual calendar of work, and assignment of duties within a school; (b) an interview with each school counselor; (c) an interview with each school principal; (d) a one-day log of counselor activity; and focus groups of (e) parents, (f) students, and (g) teachers in each school.

As evaluators designed the instrumentation, they drew on their previous knowledge of school counseling practices and their experience collecting data from constituent groups in schools. In her first meeting with the counselors, which occurred in August before the school year began, the primary evaluator asked counselors for the general areas of concern. Their remarks formed the basis for many of the areas covered in the interview guides developed by the evaluators. The initial meeting with the 13 counselors prior to beginning the evaluation was, at best, guarded. Then, themes began to emerge which were developed into interview and focus group protocols: (a) number and method of assignment of students; (b) clerical assistance and technical support; (c) relationships with administrators, parents, and teachers; (d) policies related to the counseling program; (e) administrative supervision of counselors and counseling program; (f) communication with building-level and district-office administrators; (g) job expectations and priorities; (h) amount and quality of time spent with students, parents, teachers, and other people (that is, tasks dealing directly with people rather than paper); (i) noncounseling duties; (j) others' understanding of the school counselors' job; (k) important job functions and barriers to performing these; (l) clarity of job expectations and performance evaluation; (m) helps and hindrances to doing the job well; (n) desired changes; (o) sources of frustration in the job; and (p) desirable staff development activities.

All of the areas mentioned by the superintendent and his staff eventually emerged from the counselors' discussion and formed the basis for individual interview guides for counselors and administrators as well as focus group interview guides for parents, students, and teachers. The counselor interview guide also included the following open-ended question: "What else would you like to tell me about being a high school counselor in--County?" A panel of three experienced high school counselors from other school districts critiqued the questions on the interview guides for the various groups and made suggestions for clarity before the interview guides were used. Principals responded to essentially the same questions on a parallel form of the interview guide, permitting comparison of administrator and counselor responses to similar items. A professor of educational administration critiqued this interview guide for appropriateness and clarity.

Guiding questions for the separate focus groups of students, teachers, and parents followed the same general topic areas outlined above, but were geared to the specific population being queried. Each group, for example, was asked which counselor was assigned to the student (or to the teacher's students) and how a student arranged to see the counselor. All groups answered how often they had seen the counselor and how helpful the experience was. They responded to reasons they thought students saw the counselors. They were also queried on kinds of help they would like to see available as well as areas of the counseling program which were not helpful to them. They, too, had an open-ended question ("What else would you like to tell us?"). Focus groups of students, parents, and teachers used similar interview guides so answers could be compared among the groups.

The counselor log sheet was developed by the evaluators and reviewed by three veteran high school counselors who suggested items and clarified terminology. Evaluators consulted various resources for ideas on how to construct a log form (Gysbers & Henderson, 1994; Schmidt, 1991; VanZandt & Hayslip, 1994), gathering general ideas but not finding a ready-made form that seemed to suit their purposes. As a result, the actual form used for this evaluation was based on one used successfully for several years in a school system where one of the evaluators had previously supervised school counselors. The form was revised and updated to meet current demands on counselors' time such as time spent in using c-mail. This log sheet was chosen in part because counselors who had used it previously had reported that it did not require a great deal of time to complete, and they believed they could accurately report their activities in the categories listed. Counselors were asked to complete the log sheet for one day during one specified week "on as typical a day as possible" and mail it back to the evaluators by December 4. They were asked to place a tally mark each 10 minutes to indicate what they were doing at that time. Headings were: Counseling and Advising, Coordinating, Administrative, Clerical, Testing, Teaching, Professional Development, and Working on the Telephone (incoming and outgoing categories). There was also an "Other" blank so counselors could add any activities they could not categorize under the listed headings. The intent was to make the log inclusive and easy to use, yet detailed enough to collect data that could be compared to counselors' responses, such as estimates of time and activities, previously collected in the interviews. A cover letter explained how to mark and return the log. All counselors completed and returned the log sheets within the requested time period.

The initial proposal to the superintendent also called for a survey of all teachers, students, and parents in the high schools. The evaluators intended to develop surveys from information derived from focus groups and interviews; however, so much data were collected during focus group and individual interviews that the evaluators believed they had a clear picture of the situation. In part, this may have happened because focus groups have the capacity to harness the power of human interaction by capitalizing on relationships and consequently generating insights that might otherwise remain unspoken (Creswell, 1994). Realizing that distributing and collecting surveys would take a great deal of counselor and administrative time in each school, the evaluators explained their work at that point to the superintendent and suggested revising the procedure and consequently reducing the project budget to eliminate the school-wide surveys. School district personnel agreed to this modification. Had the evaluators not heard the same content themes repeated from the individuals and groups, they would have proceeded with the survey as planned.

Collecting the Data

The primary evaluator conducted the counselor interviews privately. She and one of the doctoral students co-facilitated the nine focus groups, one each for parents, students, and teachers at each of the three high schools. Greenbaum (2000) identified several functions essential to the role of the focus group moderator: strategic consultant and planner, content manager, facilitator, analyst and communicator, and even "friend," to guide participants in the sharing of emerging data. The evaluators believe they fulfilled all of these roles at various times during the focus groups.

For reasons of efficiency, counselors arranged the separate focus groups. The evaluators requested they assemble as great a cross-section of parents, students, and teachers as possible. Practical limitations, however, dictated to some degree the composition of the groups. For example, only parents who did not work or could take time from their jobs were able to come to the school during the day. The teacher groups consisted of teachers who were free during a given period. Only students who were in nonstandards-tested classes at the appointed discussion hour were included in the groups. However, the evaluators heard a wide range of views from each group. All contained some supporters and some detractors of the counseling program, suggesting that counselors were conscientious in constituting the groups. Student responses, for example, ranged from never having met the counselor to one student believing the counselor had saved her from suicide. Parent and teacher responses also varied widely.

During the focus groups, each participant was given a copy of the interview questions. After an opening explanation, the two evaluators conducting the group took notes and asked clarifying questions. Participants were eager to talk and generally needed no prodding. On a few occasions, noncontributing students or parents were directly asked by the evaluators to comment on a question, and in every instance, they added their comments.

Evaluators spent one day at each high school conducting individual interviews for counselors and principals and three focus groups: students, parents, and teachers. Additionally, at some schools others (including guidance secretary, librarian, or teachers, parents, and students who wanted to add something private after the focus groups) also wanted to talk to the evaluators. At one school, the counselors at their request also met as a group and talked with the evaluators. In every school, someone in a focus group returned later to make additional, usually positive and very personal comments. The evaluators also ate lunch at each school cafeteria in order to observe interactions, including students and teachers informally interacting with counselors, to gain general impressions. The doctoral assistant sat in the reception area of the counseling office for a brief time to see and hear what she might. While no additional data were formally collected during these times, evaluators tried to gain some feeling of the climate of the counseling offices and the frequency of student, teacher, and parent traffic as well as observing materials available for student and parent use.

Organizing and Reporting the Data

Data were reported in six sections (Counselor Interviews, Counselor Log Sheet Data, Principal Interviews, Teacher Focus Group Interviews, Student Focus Group Interviews, and Parent Focus Group Interviews) followed by a Conclusions and Recommendations section. Most of the report was in narrative form except for activities logged by the counselors, which were reported as percentages of their workday. Evaluators organized the data around recurring themes that emerged from an analysis of responses given by counselors, parents, students, teachers, and administrators in individual interviews and focus groups. One of the evaluators listed interviewees' and group members' responses and organized them into categories as the frequency of the recorded comments suggested. One example of how counselor comments were used is illustrated by the following statement from the report, substantiated by a counselor's comment: "Counselors at one school report inconsistent responses from administrators; they believe they are assigned many administrative tasks such as setting up meetings and filing. One counselor noted, 'Should have started seeing our seniors and we haven't ... hard to find time to see the students.'" While comments were used throughout the final report, evaluators took care that the quotes were general enough not to be attributable to an individual, honoring their promise of confidentiality to the counselors.

Information from counselor log sheets was compiled for each school and then for the district as a whole, reported by category as percent of the total day (37%, counseling; 26%, clerical; 16%, testing; 9%, coordinating; 9%, administrative; 2%, professional development; 1%, teaching). The percentages were accompanied by a narrative analysis with further breakdown of activity such as the following: "More time was logged in the Counseling and Advising category than any other ...; all counselors recorded activity in this area.... Most of the counseling contacts (69%) involved academic and college counseling advising; only one contact was recorded for crisis intervention, and 13% of the contacts were for personal counseling ... No counselor reported doing any group counseling.

The evaluators reviewed documents furnished to them regarding the counseling program such as minutes of counselors' meetings and student handbook descriptions of counseling. Little was found in these documents to necessitate comment in the final report. There was no job description, counseling manual, or program of work. As a result, the final report contained recommendations to develop these.

The Conclusions and Recommendations section consisted of 27 findings, each of which was followed by a clear and measurable recommendation. The following is an example of a finding and the corresponding recommendation:

Finding: There is a good bit of variation in practices and procedures among the three high schools, although the basic services are the same.

Recommendation: Hold periodic, perhaps monthly, meetings of counselors to share ideas and procedures. Be sure at least part of the meeting time is for high school counselors to meet together. This time can also be used to develop grade packets and other materials, to avoid duplication of effort among the schools [referring to another recommendation]; to suggest policy and develop procedures; to vent frustrations and seek solutions; and to provide skills training for counselors.

The report was written with a lay audience in mind, with every effort to use clear, jargon-free language. The recommendations were deliberately not prioritized; however, the report would have been stronger if it had been organized as long-term and short-term recommendations, something the lead evaluator realized after making oral presentations and answering questions about the data.

The report also included as appendices copies of the interview guides, counselor log sheet and letter of instructions, the National Standards for School Counseling Programs (ASCA, 1999a; Campbell & Dahir, 1997), the ASCA (1999b) Role of the Professional School Counselor, and a sample plan of work, with activities arranged by month. This plan of work, requested by the superintendent, was patterned after a document prepared by one of the high schools, but also included items based on the recommendations of the evaluation report such as duties to be completed during the summer months. The following is an example of a finding and recommendation based on the National Standards for School Counseling Programs

Finding: There appears to be no comprehensive plan for working with individual students; counselors perform tasks but do not seem to have any over-arching plan for what they accomplish with each student.

Recommendation: Develop a plan of approach for each student emphasizing academic, career, and personal/social development as suggested in the ASCA National Standards for School Counseling.

An example of a finding and recommendation referencing the Role of the Professional School Counselor (ASCA, 1999b), follows.

Finding: Almost all counselors have a higher number of students than they can serve effectively. They currently spend a relatively low proportion of their time in counseling.

Recommendation: Reduce the counselor/student ration to no more than the 350/1 set by [state]. Plan to reduce counseling loads to the 250/1 ratio recommended by the American School Counselor Association.

The evaluator made three presentations of the data: to school administrators and central office personnel; to the high school counselors; and to the school board. The administrators asked a few questions about the data; the school counselors were mostly interested in the implementation of the recommendations (which could not be addressed by the evaluators); and the school board members asked extensive questions. The comments and questions of the latter group required the evaluator to answer from her experiences with other school districts and to draw some comparisons with school counseling in general, in order to give perspective on this district's strengths and problems. Each administrator, counselor, and school board member received a copy of the 68-page report.

Following up the Evaluation

About 3 months after the evaluation was concluded, the evaluators asked the counselors, administrators, and central office personnel to complete a simple questionnaire to get their reactions to the process. All counselors and principals and some central office personnel completed and returned the questionnaire. In answer to the question, "How intrusive was the evaluation process to your work?" all answered "not very intrusive" except one counselor who noted "somewhat intrusive." Almost all indicated they thought they were able to express their views "very fully" with one adding, "felt comfortable to vent." In response to the question, "How well do you believe the report showed the situation as it exists?", all but one said "quite well" and the one person checked "somewhat." Surprising to the evaluators, all but one person indicated they "agree with most recommendations," the highest choice on the four-item response scale. Thus, evaluators believe they represented the viewpoints of the various constituents and made logical recommendations from the point of view of the school personnel.

There was less agreement from the participants on what they believed would happen following the report. In response to the question "How confident are you that things will change based on this evaluation?", the responses were mixed: 16.5% indicated they believe change is unlikely, 67% expected "some change maybe," and 16.5% indicated they believe "change is likely." In actual fact, some changes were made within a few months following delivery of the report. One male was hired as a school counselor, a beginning at following the recommendation to make the previously entirely-female secondary counseling staff more diverse. A person with expertise and experience in counseling has been appointed to supervise counselors at the district level, fulfilling one of the report recommendations. Counselors were employed for additional days during the summer, a very strong recommendation in the report. A commitment has been made to have periodic meetings to improve communication among counselors, school administrators, and district level administrators.

In response to the request "What suggestions would you make for another program evaluation of this type?" and "Other comments," participants noted, "Nothing was said that hasn't been said by counselors themselves for years," and "I think this was done efficiently and produced an excellent picture of the county program. However, I expect absolutely no results that depend on board or administrative approval." One noted, "Everything was great. The time-line was reasonable and the recommendations given were satisfying." One counselor wrote, "Would have liked more definite information as to the [use of] the results; no one at central office seemed to be able to tell us what would happen once the report was finished--even if counselor would see it!" Another said, "I liked the evaluation process as conducted. Very thorough!"

LEARNING FROM THE EXPERIENCE

Others undertaking such a project will need to recognize that the rigors of good research may not fit with the timely needs of a school district for information. This was a very time-consuming project for the evaluators and, therefore, of some cost to the school district. Because the district wanted the information fairly quickly, the evaluators were not able to collect data over as long a time span as would have been desirable. Not drawing a random sample of teachers, parents, and students may have compromised the results of the evaluation. However, because this evaluation emphasized the program evaluation process specifically, the evaluators decided that timeliness, cost, and minimal intrusion of already overburdened counselors and administrators outweighed more rigorous research methodology. A survey of all parents, students, and teachers would have provided more data, but would have added a substantial increase in cost and time.

The evaluators discovered that comparisons between groups were important, verifying the usefulness of parallel forms of the interview guides. For example, the principals' and counselors' estimates of how counselors spent their time varied widely, and the counselor logs showed yet another set of numbers. The principals reported that they had evaluated the counselors; the counselors reported that they had not had a performance evaluation. The evaluators did not attempt to find the truth in these statements, instead simply reported the discrepancies. Those undertaking such a program evaluation must be prepared for the inevitable finger-pointing and accusations that such discrepancies may engender and must, particularly if external evaluators, stay removed from the fray and suggest solutions to revealed problems.

In an ideal world, an evaluator might shadow each counselor for a day or two and gather observational data. The technique was rejected for this evaluation because it was too intrusive, but counseling interns, for example, might be able to accomplish this in a way that external evaluators could not. External evaluators from a university may find this an ideal way to offer service to a school counseling program seeking evaluation.

In this particular evaluation, there was very good cooperation because all parties indicated they thought counseling should be improved, and most people thought the counselors were well-prepared and competent. No group interviewed, for example, expressed concerns about confidentiality or quality of service; they wanted a counseling program which better met the needs of the students. An evaluation of this type would be markedly different if the administrative staff had a pre-conceived agenda or if, in fact, the counselors were poorly prepared or indifferent.

It was fairly easy to establish rapport and credibility in this instance because the primary evaluator had many years of both school counseling and district level administrative experience as well as some previous history of consulting with other administrators of this particular district. Establishing rapport and trust to obtain honest and complete answers is essential in this kind of evaluation in order for various viewpoints are to be represented. In addition, if feasible recommendations are to be made, the person(s) evaluating and presenting the data must have sufficient background and knowledge to be able to understand both administrative and counseling viewpoints. To make feasible recommendations in this district, for example, one must understand the requirements and the frustrations currently imposed by the state's high-stakes testing program at the high school level as well as the nature of transition from a rural to a suburban school district. School districts planning to conduct such an external evaluation will be well-advised to select evaluators who know not only how to conduct a program evaluation, but who also understand the realities of life in a public school district. Ideally, the evaluators will have had some experience working with school boards as well as with counselors and administrators. External evaluators should take care to establish their own credibility and must be able to understand the internal workings and external forces impacting on the system they are evaluating

School counselors, busy as they are, can help themselves in such an external audit by cooperating fully and by taking every opportunity to present their job activities and their counseling program in ways that describe their work situation as completely as possible. It is likely that in most program evaluations, conditions exist that impact the counselors' jobs negatively but are beyond their control. Conscientiously helping to collect data can promote a clearer understanding of the frustrations of a school counselor's daily life: the interruptions; the ever-increasing time demands; and the constant feeling of being in the middle among the demands of administrators, district office personnel, parents, teachers, and students. The more data a counselor can provide about daily and yearly work, the clearer the overall picture will be. Having an ongoing record of one's daily work in some organized form can be a great help to everyone when external evaluators enter the school.

Perhaps school counselors and school administrators will someday have the luxury of time to keep ongoing records such that accountability can be established without an external audit. Until that time, external evaluators can provide a valuable service as they make recommendations for improvement of school counseling programs.

References

American School Counselor Association. (1998). Ethical-standards for school counselors. Retrieved March 14, 2002, from http://www.schoolcounseling.org/content. cfm?L1=1&L2=15

American School Counselor Association. (1999a). National standards. Retrieved March 14, 2002, from http://www. schoolcounselor.org/content.cfm?L1=1&L2=9

American School Counselor Association. (1999b). The role of the professional school counselor. Retrieved March 14, 2002, from http://www.schoolcounselor.org/content. cmf?L1=1000&L2=69

Campbell, C. A., & Dahir, C.A. (1997). Sharing the vision: The national standards for school counseling programs. Alexandria, VA: American School Counselor Association.

Creswell, J. W. (1994). Qualitative inquiry and research design: Choosing among five traditions. Thousand Oaks, CA: Sage.

Fitch, T., Newby, E., Ballestero, V., & Marshall, J. (2001). Future school administrators' perceptions of the school counselor's role. Counselor Education and Supervision, 41, 89-100.

Greenbaum,T. L. (2000). Moderating focus groups: A practical guide for group facilitation. Thousand Oaks, CA: Sage.

Gysbers, N., & Henderson, P. (1994). Developing and managing your school guidance program (2nd ed.). Alexandria, VA: American Counseling Association.

Hayes, R. L., Paisley, P. O., Phelps, R. E., Pearson, G., & Salter, R. (1997). Integrating theory and practice: Counselor educator-school counselor collaborative. Professional School Counseling, 1(1), 9-12.

Herr, E. L. (1979). Guidance and counseling in the schools: Perspectives on the past, present, and future. Falls Church, VA: American Personnel and Guidance Association.

Hogan, C. C. (1998). Integrate high-school counselors into the learning process. Education Digest, 64(1), 55-58.

Kramer, G. L. (1992). Using student focus groups to evaluate academic support services. NACADA Journal 12(2), 38-41.

Lombana, J. H. (1985). Guidance accountability: A new look at an old problem. The School Counselor, 32, 340-346.

Morrow, G. R, Burris-Kitchen, D., & Der-Karabetian, A. (2000). Assessing campus climate of cultural diversity: A focus on focus groups. College Student Journal, 34, 589-602.

Mueller, M. R., & Krueger, R. A. (1986, July). Qualitative evaluation from a utilization focused perspective: The Minnesota experience. Paper presented at the joint meeting of the Evaluation Research Society and Evaluation Network, Toronto. (ERIC Document Reproduction Service No. ED267104).

Perusse, R., Goodnough, G. E., & Noel, C. (2001). A national survey of school counselor preparation programs: Screening methods, faculty experiences, curricular content and fieldwork requirements. Counselor Education and Supervision, 41, 252-262.

Schmidt, J. (1991). A survival guide for the elementary/middle school counselor. West Nyack, NY: The Center for Applied Research in Education.

Schmidt, J. (1995). Assessing school counseling programs through external reviews. The School Counselor, 43, 114-123.

Studer, J. R., & Sommers, J. A. (2001). The professional school counselor and accountability. NASSP Bulletin, 84(615), 93-99.

VanZandt, C. E., & Hayslip, J. B. (1994). Your comprehensive school guidance and counseling program. New York: Longman.

Claire Cole Curcio, Ed.D., LPC, NCC, is a Professor Emerita, and Christina Mathai, Ph.D. is a recent graduate of Counselor Education at Virginia Tech, Blacksburg, VA. E-mail: [e-mail]

Jane Roberts, Ph.D., MSW, LCSW, ACSW, is a Health Sciences research specialist, Durham (NC) Veterans Administration Hospital.