Evaluating a small-group counseling program--a model for program planning and improvement in the elementary setting.
As educational leaders have increased the focus on educational outcomes, groups are looking at the connection between learning and mental health. For example, the UCLA Center for Mental Health in Schools (Adelman & Taylor, 2002) published a report that identified mental health problems of students as one of the key barriers to learning in our nation's schools. It concluded that remedying the barriers hindering performance and learning is critical to increasing academic performance. Counseling and guidance programs are critical in addressing mental health problems that interfere with a student's ability to learn, to succeed, and to participate in the learning process (Lockhart & Keys, 1998). School counselors provide services to students already identified with mental health problems and routinely engage in activities that prevent the mental health problems (Steen & Kaffenberger, 2007).
Children who experience difficulties in their peer relationships exhibit concurrent behavioral, psychological, and academic problems and are at heightened risk for a wide variety of later maladjustment (Parker, Rubin, Erath, Wojslawowicz, & Buskirk, 2006; Rubin, Bukowski, & Parker, 2006). Social skills intervention not only improves social functioning but also improves future mental health and academic performance (Asher, Parker, & Walker, 1996; Consortium on the School-Based Promotion of Social Competence, 1996). Although school counselors commonly lead social skills groups, little attention has been given to how effective these interventions are in school settings and how school counselors should evaluate them.
The No Child Left Behind Act (NCLB; U.S. Department of Education, 2001) mandates schoolbased accountability and has impacted school counselors' roles in two ways. First, NCLB's focus on academic outcomes (i.e., standardized test scores) has stimulated professionals' interests in nonacademic factors such as social-emotional learning and its connection to academic outcomes. Second, NCLB's demand for accountability emphasizes that school programs, including school counseling, should document effectiveness (Dahir & Stone, 2003). Paralleling NCLB, the ASCA National Model[R] (American School Counselor Association, 2005) emphasizes that program evaluation is part of a comprehensive school counseling program. This implicitly underscores that documenting the effectiveness of the social-emotional components of a school counseling program is important. The urgency for school counselors to become more accountable is creating a clear set of new expectations for all school counselors. Stone and Dahir (2007) have encouraged school counselors to learn how to build accountability standards into counseling programs. Accountability can be directly incorporated into a comprehensive school counseling program.
Accountable school counseling programs have several key components, as presented in the following:
1. School counseling programs should be developed according to assessed needs at the school (Johnson & Johnson, 2003).
2. School counselors should select interventions that are evidence based (Carey & Dimmitt, 2006).
3. School counselors should use valid and reliable assessment procedures to evaluate the effective ness of their activities (Steen & Kaffenberger, 2007; Studer, Oberman, & Womack, 2006).
4. As school counselors collect data, they also must interpret the data and use it for program improvement and to inform their practice (Rowell, 2006).
By using these elements, school counselors can build effective program evaluation into the planning and implementation process.
OBSTACLES TO ACCOUNTABLE PRACTICE
School counselors sometimes view accountability as threatening in that positive results may lead to additional assignments in an already overburdened profession (Studer et al., 2006), and many school counselors do not have the necessary skills to conduct program evaluation or research. To assist school counselors to conduct program evaluation and research, new materials (Stone & Dahir, 2007) and professional development opportunities (Rowell, 2006) are needed. School counselors also have time constraints and extra duties that compete with their expectation to implement and measure the effectiveness of counseling interventions (Dollathide & Lemberger, 2006).
The expectations, challenges, and limited support for school counselors to increase their accountability and directly engage in program evaluation and research can be daunting for the average practitioner. One way to approach this task is to evaluate each component (such as individual counseling, smallgroup interventions, and classroom guidance) of the larger school counseling program (Steen & Kaffenberger, 2007). By evaluating individual components, school counselors can select the most effective components for inclusion in a comprehensive counseling program. This article shows how a school counselor conducted a feasible and effective program evaluation of a small-group counseling program in a manner consistent with best practices in accountability.
All of the students in the third grade over a 3-year period were given a self-report screening instrument measuring loneliness and social anxiety. The teachers and school counselor identified the students who had the lowest scores on the screening instrument as students in need of small-group counseling. Based on the results of the student screening instrument, 49 third-grade students were recommended for small-group social skills intervention. Fifty-one percent of the students were female. The ethnic distribution of the students was the following: White, 66%; Hispanic, 26%; and African American, 8%.
To maximize the likelihood of measurable positive student outcomes, an evidence-based group counseling program entitled Social Skills Group Intervention, or S.S.GRIN (DeRosicr, 2002, 2007), was selected for implementation. In a previous study of third graders, children with specific peer problems (high peer dislike, victimization by peers) and/or high social anxiety were identified and randomly assigned to S.S.GRIN groups administered by school counselors. Two studies, employing a randomized experimental control group &sign, support the effectiveness of S.S.GRIN (DeRosier, 2004; DeRosier & Marcus, 2005).
Children who participated in S.S.GRIN showed improvement in peer liking, enhanced self-efficacy for dealing with social situations, lower social anxiety, and greater declines in aggressive behavior problems compared to children in the control group (DeRosier, 2004). At one-year follow-up, all pretest and posttest benefits were still present and additional benefits were evident, including lower victimization by peers, fewer negative nominations by peers, and more positive outcome expectancy for social situations (DeRosier & Marcus, 2005). To determine if similar benefits would be evident when implemented outside a major research study and to meet accountability requirements (Carey & Dimmitt, 2006), a program evaluation was conducted by the school counselor.
S.S.GRIN (DeRosier, 2002, 2007) is a highly structured, cognitive-behavioral intervention with detailed session scripts and activities. Sessions focus on the development of verbal and nonverbal communication, initiation, cooperation, compromise, and negotiation skills. In addition, emotional regulation, impulse control, and perspective taking are emphasized. Skills for coping with teasing and peer pressure also are developed. Each session combines skill-based instruction with a variety of role playing, modeling, and hands-on activities. The topics covered in S.S.GRIN address personal/social objectives in the ASCA national standards for students (ASCA, 2005).
The teachers and school counselor assigned five to six students (keeping in mind a balance of girls/boys and ethnicity) in each S.S.GRIN group to participate in the 10-session program. Ten groups were held over 3 years. Although teacher input was requested, student perceptions of their need for intervention were strongly weighted when selecting students. The group process can be more effective if the student feels a need to improve his or her friendship skills. In addition, student self-referrals, as compared to teacher referrals, tend to be less biased toward over-referral of boys for intervention (Davis, 1978). In the school counselor's prior experience, when teachers identified students they perceived as needing to improve their friendships without student self-report, many of the identified students needed intervention in behavior or anger management prior to interventions for friendship skills.
The identified students were invited to participate in the groups by the school counselor. All 49 students gave verbal assent to participate in the groups, and then parent consent was obtained through permission letters sent home. Parents were given the school counselor's phone number on the form and encouraged to call if they had questions or wanted more information.
S.S.GRIN includes a token economy behavior management system with short- and long-term rewards for appropriate participation and behavior. In addition, the group members added rules to the standard S.S.GRIN rules during the first session. To help the participants take a personal ownership in the group, the school counselor had each participant perform a randomly selected job each week (e.g., rule reader and reminder, compliment provider, and summarizer). Following the intervention, the students were encouraged to take their group portfolio home to update their parents on group topics.
Two areas of emotional adjustment were assessed: loneliness and social anxiety. Both constructs were assessed using subsets of items from larger, published scales to meet time demands and minimize student fatigue. Loneliness was measured with a subset of eight items from the 16-item Loneliness and Social Dissatisfaction Questionnaire (Asher & Wheeler, 1985). The scale has excellent reported reliability, and its utility for assessing children?s loneliness in the school social context has been repeatedly demonstrated (Asher & Wheeler; Crick & Ladd, 1993). Although loneliness was assessed using only a subset of the original items in this study, the subset had comparable internal consistency to the full scale ([alpha] = .83). Social anxiety was measured with a subset of five items from the 18-item Social Anxiety Scale for Children-Revised (La Greca & Stone, 1993). La Greca and Stone reported adequate internal consistency for the scale, and concurrent validity was demonstrated in that children with problematic peer relations reported higher levels of social anxiety. The subset of items used in this study had adequate internal consistency ([alpha] = .76). For each child report measure, scores on the items were summed and averaged to give a total scale score, with higher scores indicative of higher loneliness or social anxiety.
Pre- and post-assessments of emotional adjustment (child-reported) and academic achievement (standardized test scores) for the 49 students were compared to assess the effectiveness of the smallgroup counseling program. Academic achievement was measured using end-of-grade (EOG) standardized test scores for reading. The scores are designed to assess the child's degree of mastery of topics covered in the North Carolina Standard Course of Study for reading. Descriptive information (i.e., means, standard deviations) for each measure at both time points is presented in Table 1.
Paired-sample t tests were conducted to determine change in loneliness, social anxiety, and reading scores related to group participation (see Table 2). Statistically significant reductions in loneliness (t = -4.95, p < .001, d = -.88) and social anxiety (t = -4.06, p < .001, d = -.64) were evident. In addition, a statistically significant increase in EOG reading scores (t = 11.89, p < .001, d = .92) was found. As well as statistically significant, the results appeared clinically significant, as evidenced by the magnitude of the effect sizes.
Results should be viewed with caution, however, because the study did not include a control group. Consequently, no strong conclusions can be drawn in reference to the magnitude of the observed changes for participants as compared to the normal patterns of change in loneliness, social anxiety, and reading during the time interval.
After the initial study, the school counselor continued to use the S.S.GRIN program and found that third-grade students who indicated they were lonely and worried about friendships through the self-assessment measure rated themselves as less worried about friendships after completing the small-group experience. Also, the majority of participants' EOG post-scores were higher than the pretest scores. After the completion of the small-group program, the school counselor sent teachers and parents an informal evaluation asking if they thought their participating child seemed to be more cooperative, have more confidence, and display more effective communication skills. The majority of responses from parents and teachers were positive. In fact, the parents and teachers seemed to see more student improvement in social skills than did the children themselves. Because of the continuing success of this small-group experience, the school counselor uses this small-group program as a part of a three-tiered approach that addresses social skills development in the third grade:
1. The school counselor presents social skill lessons in the third-grade classrooms.
2. Students who then self-assess as deficient in social skills, or whose teacher or parent identifies their social skill deficiency, can participate in the social skills groups.
3. After group completion, the school counselor works individually with those students who continue to have social skills issues.
IMPLICATIONS FOR SCHOOL COUNSELORS
As school counselors plan their programs to support accountability, there are several considerations that arise in the study presented here. Program evaluation can be easily incorporated into routine counseling activities when the following implications for school counselors are addressed:
1. Student needs are assessed through screening instruments in combination with teacher and parent consultation. When interventions are based upon what children need, they are more likely to be effective.
2. Interventions are selected from evidence-based programs. The new emphasis on programs that have supporting research reflects the greater effectiveness of program initiatives when they include evidence-based interventions.
3. Simple, but valid and reliable, assessments are used to measure impact. When such assessments are used, the school counselor is more likely to get results that are accurate and measure real change.
4. Assessments are measured pre- and post-intervention. The measurement of change is only possible when both pre- and post-intervention assessments are conducted.
5. Results are shared with stakeholders and used to plan future school counseling initiatives. School counselors who share their results are seen by educational peers and the educational community as contributing to meet overall school goals and expectations. They also show a commitment to change and alter programs to focus on what is having an impact, rather than continuing to do things without knowing if they help students.
Counselors' stakeholders (i.e., school and district administrators and counselor educators) should take several steps to fully support school counselors in their efforts to be accountable and to demonstrate measurable impact. Principals should encourage research-based program planning and ask for program evaluation results to support current practices. To support program evaluation and implementation of small-group interventions, administrators will need to prioritize these activities over non-counseling functions. District administrators also can encourage research-based program planning by offering training in research methods and data-based decision-making. They also can set collective expectations for program planning and assist in this process. Finally, counselor educators can review their curricula to determine if their coursework reflects current expectations for accountability and research-based program planning. When school counselors are supported in their efforts to show how they contribute to student success at school, they will focus on improving their programs based on student outcomes in a manner now expected of teachers and other educators.
As school counselors face increasing pressures to be accountable for outcomes such as test scores, program evaluation needs to be directly incorporated into models of program planning. School counselors face two major barriers when it comes to implementing effective counseling interventions on behalf of the students in their schools. First, because the evaluation of effectiveness is necessarily evidence based, there is a need for school counselors to prove that their interventions make a discernable difference and help children achieve in school. Second, school counselors are often given non-counseling responsibilities (testing coordinator, special education referral team leader, administrator of retests, etc.) that reduce the time available for small-group interventions.
In this article, a school counselor conducted a program evaluation of a small-group intervention to demonstrate her impact on students and also to inform her practice. The small-group intervention was nested within a system of classwide intervention, screening, and individual student supports, as recommended in the delivery system of the ASCA National Model (2005). Specifically, classroom lessons on social skills were first provided, and then follow-up needs assessments with teacher and parent feedback were conducted to identify students in need of more intensive, small-group social skills intervention. This model of service delivery incorporated program evaluation to show the impact of social skills training on students and support the use of a research-supported counseling intervention in the school counselor's specific school setting. At the same time, this approach enabled the school counselor to determine if the small-group social skills intervention should continue to be included as part of her comprehensive school counseling program.
In this study, students were identified as having difficulty in social skills, interfering with their ability to learn at school. To address these needs, the school counselor selected a research-based small-group counseling intervention (S.S.GRIN) with evidence of program impact (DeRosier, 2004; DeRosier & Marcus, 2005). To ensure that similar results were found in her particular school setting, the school counselor conducted pre- and post-intervention surveys that were easy to administer. By conducting the program evaluation, the school counselor was able to show stakeholders (i.e., principals, teachers, and parents) that her efforts had a measurable impact on students' levels of loneliness, social anxiety, and reading achievement.
Adelman, H., & Taylor, L. (2002). Building comprehensive, multifaceted, and integrated approaches to address barriers to student learning. Childhood Education, 78, 261-268.
American School Counselor Association. (2005). The ASCA national model: A frame work for school counseling programs (2nd ed.). Alexandria, VA: Author.
Asher, S. R., Parker, J. G., & Walker, D. L. (1996). Distinguishing friendship from acceptance: Implications for intervention and assessment. In W. M. Bukowski, A. F. Newcomb, & W. W. Hartup (Eds.), The company they keep: Friendship in childhood and adolescence (pp. 366-405). New York: Cambridge University Press.
Asher, S. R., & Wheeler, V. A. (1985). Children's loneliness: A comparison of rejected and neglected peer status. Journal of Consulting and Clinical Psychology, 53, 500-505.
Carey, J. C., & Dimmitt, C. (2006). Resources for school counselors and counselor educators: The Center for School Counseling Outcome Research. Professional School Counseling, 9, 416-420.
Consortium on the School-Based Promotion of Social Competence. (1996). The school-based promotion of social competence. In R.J. Haggerty & L. R. Sherrod (Eds.), Stress, risk, and resilience in children and adolescents: Processes, mechanisms, and interventions (pp. 268-316). New York: Cambridge University Press.
Crick, N. R., & Ladd, G. W. (1993). Children's perceptions of their peer experiences: Attributions, loneliness, social anxiety, and school avoidance. Developmental Psychology, 29, 244-254.
Dahir, C. A., & Stone, C. B. (2003). Accountability: A M.E.A.S.U.R.E of the impact school counselors have on student achievement. Professional School Counseling, 6, 214-220.
Davis, W. E. (1978). A comparison of teacher referral and pupil self-referral measures relative to perceived school adjustment. Psychology in the Schools, 15, 22-26.
DeRosier, M. E. (2002). Group interventions and exercises for enhancing children's communication, cooperation, and confidence. Sarasota, NY: Professional Resources Press.
DeRosier, M. E. (2004). Building relationships and combating bullying: Effectiveness of a school-based social skills group intervention. Journal of Clinical Child and Adolescent Psychology, 33, 125-130.
DeRosier, M. E. (2007). Social skills group intervention (S.S.GRIN): Group interventions and exercises for enhancing children's communication, cooperation, and confidence (4th ed.). Cary, NC: 3-C Institute for Social Development.
DeRosier, M. E., & Marcus, S. R. (2005). Building friendships and combating bullying: Effectiveness of S.S.GRIN at one-year follow-up. Journal of Clinical Child and Adolescent Psychology, 34, 140-150.
Dollarhide, C., & Lemberger, M. (2006). "No Child Left Behind": Implications for school counselors. Professional School Counseling, 9, 295-304.
Johnson, S., & Johnson, C. D. (2003). Results-based guidance: A systems approach to student support programs. Professional School Counseling, 6, 180-184.
La Greca, A. M., & Stone, W. (1993). Social Anxiety Scale for Children-Revised: Factor structure and concurrent validity. Journal of Clinical Child Psychology, 22, 17-27.
Lockhart, E. J., & Keys, S. G. (1998).The mental health counseling role of school counselors. Professional School Counseling, 1, 3-6.
Parker, J. G., Rubin, K. H., Erath, S. A., Wojslawowicz, J. C., & Buskirk, A. A. (2006). Peer relationships, child development, and adjustment: A developmental psychopathology perspective. In D. Cicchetti & D.J. Cohen (Eds.), Developmental psychopathology, Vol, 1: Theory and methods (2nd ed., pp. 96-161). New York: Wiley.
Rowell, L. L. (2006). Action research and school counseling: Closing the gap between research and practice. Professional School Counseling, 9, 376-384.
Rubin, K. H., Bukowski, W., & Parker, J. (2006). Peer interactions, relationships, and groups. In N. Eisenberg (Ed.), Handbook of child psychology: Social, emotional, and personality development (6th ed., pp. 571-645). New York: Wiley.
Steen, S., & Kaffenberger, C. J. (2007). Integrating academic interventions into small group counseling in elementary school. Professional School Counseling, 10, 516-519.
Stone, C. B., & Dahir, C. A. (2007). School counselor accountability: A MEASURE of student success (2nd ed.). New York: Prentice Hall.
Studer, J. R., Oberman, A., & Womack, R. H. (2006). Producing evidence to show counseling effectiveness in the schools. Professional School Counseling, 9, 385-391.
U.S. Department of Education. (2001). The No Child Left Behind Act of 2001. Pub. L. No. 107-110.
Dee Bostick, M.A., Ed.S., is an elementary school counselor in Wake County Public Schools, NC. E-mail: firstname.lastname@example.org Ron Anderson, Ph.D., is vice president of SELmedia, Inc., and formerly the senior director of Counseling and Student Services in Wake County Public Schools.
Table 1. Pretest and Posttest Means and Standard Deviations for All Variables Measure M SD Loneliness T1 2.24 .78 Loneliness T2 1.67 .77 Social anxiety T1 2.20 .55 Social anxiety T2 1.87 .48 EOG reading T1 239.92 9.73 EOG reading T2 248.78 9.52 Table 2. Results of Paired-Sample t Tests for Intervention Effects Variable Difference SD t df p d Loneliness -.33 .47 4.95 48 < .001 -.88 Social anxiety -.34 .58 11.06 48 < .001 -.64 EOG reading 8.86 5.22 11.89 48 < .00 1 .92
|Printer friendly Cite/link Email Feedback|
|Author:||Bostick, Dee; Anderson, Ron|
|Publication:||Professional School Counseling|
|Date:||Aug 1, 2009|
|Previous Article:||The impact of positive behavior support to decrease discipline referrals with elementary students.|
|Next Article:||At-risk ninth-grade students: a psychoeducational group approach to increase study skills and grade point averages.|