Comprehensive school counseling programs and academic achievement--a rejoinder to Brown and Trusty.

In this brief article, the author responds to Brown and Trusty's assertion that based on the deficiencies of existing comprehensive school counseling program (CSCP) research, school counselors need not focus their time and effort on showing causal connections between their programs and measures of student academic achievement. Rather than defending the specific merits of the CSCP research, this rejoinder article addresses (a) the larger issue of whether Brown and Trusty have mischaracterized a key underlying results-based assumption of CSCPs, as well as (b) the value of multiple research paradigms in CSCP evaluation studies, and (c) the importance of systemic thinking and collaboration within CSCP research.

**********

In Brown and Trusty's article "School Counselors, Comprehensive School Counseling Programs, and Academic Achievement: Are School Counselors Promising More Than They Can Deliver?" comprehensive school counseling programs such as the ASCA National Model[R] (American School Counselor Association, 2003b) and their ability to promote student achievement come into serious question. Although not citing specifically where the ASCA National Model makes this pronouncement, these authors allege that intrinsic to comprehensive school counseling programs (CSCPs) is a pivotal causal assumption: When correctly planned and executed, they will lead to enhanced student achievement. Brown and Trusty go on to criticize, albeit gently, the promising body of CSCP studies for failing to meet the rigorous experimental standards needed to make causal inferences. Furthermore, the authors recommend that because the CSCP studies are too few in number and methodologically deficient to show cause and effect, it is now time for the profession to revisit this basic presupposition.

Briefly stated, therefore, Brown and Trusty are proposing a midcourse correction in the trajectory of school counseling program evaluation research, arguing instead that school counselor educators and practitioners should turn (or return) their attention to demonstrating the positive effects of small-scale (or proximal) activities, services, and interventions on academic outcomes.

Given space limitations, I cannot fully consider the various positions advanced by these authors. Accordingly, I comment here on a few key points made by Brown and Trusty. First, I reflect upon the authors' contentions that (a) the framers of CSCPs assumed they could demonstrate a cause-and-effect link between school counseling programs and student achievement; and (b) if CSCP researchers cannot produce distal causal evidence, the profession should then shift its research efforts to showing proximal relationships. Second, I defend the use of multiple and mixed research methodologies in CSCP program evaluations. In closing, I revisit the important role that collaboration and systems thinking should exert in CSCP studies.

CSCPs AND STUDENT ACHIEVEMENT

Brown and Trusty in their article are correct in affirming that many of ASCA's recent publications as well as leading scholars in the profession (e.g., Gysbers & Henderson, 2005; Paisley & McMahon, 2001) have redoubled their efforts to persuade school counselors not only to be more proactive in helping to fulfill their schools' educational missions but also to become more intentional about assisting students to reach their academic goals. In other words, school counselors ought to be more concerned with contributing to positive student academic success (Gysbers, 2003). This "push" for increased accountability also is embedded in ASCA's (2004b) role statement, whereby professional school counselors are asked to "use data to show the impact of the school counseling program on school improvement and student achievement" (Accountability section). Similarly, ASCA's (2004a) latest revision of its ethical standards includes a statement that school counselors need to assess the effectiveness of their programs in terms of enhancing "students' academic, career and personal/social development through accountability measures especially examining efforts to close achievement, opportunity and attainment gaps" (section A.9, Evaluation, Assessment and Interpretation). Undoubtedly then, by analyzing their day-to-day practice, ethical counselors will focus their efforts on explaining how their services and activities positively influence student achievement and, as a result, share responsibility for school improvement with other building educators.

The thornier question is whether the framers of these ASCA publications actually intended for school counselors to prove that their comprehensive programs lead to higher test scores and grade point averages. Brown and Trusty in their article seem to think so. After carefully perusing the ASCA National Model (2003b), I could not find any assertion that the originators' long-term goal is to generate evidence of clear causal connections at the programmatic (or distal) level to student achievement. Rather, there are phrases liberally used throughout, such as counselors are "collaborators and advocates" (p. 10), "become catalysts for educational change" (p. 15), "call attention to situations within the schools that ... hinder students' academic success" (p. 15), and use "effective strategies ... to meet stated student success and achievement" (p. 16). These imply that school counselors operating within CSCPs need to document how they assist with the academic/educational development of students. Furthermore, if the ultimate goal of CSCPs is to produce higher test scores, surely the model's Executive Summary (ASCA, 2003a) would indicate this. In fact, it only obliquely suggests that school counselors be involved in academic outcomes assessment:

   The model provides the mechanism with
   which school counselors and school counseling
   teams will design, coordinate, implement,
   manage and evaluate their programs for students'
   success. It provides a framework for the
   program components, the school counselor's
   role in implementation and the underlying
   philosophies of leadership, advocacy and systemic
   change. When implementing an ASCA
   National Model-based program, school counselors
   switch their emphasis from service-centered
   for some of the students to program-centered
   for every student. It not only answers
   the question, "What do school counselors
   do?" but requires school counselors to
   respond to the question, "How are students
   different as a result of what we do?"

Although counselors are important to the academic success of students, they are only partners in student learning and achievement (Dahir & Stone, 2003; Musheno & Talbert, 2002). It appears, therefore, that Brown and Trusty have imposed a subtext to CSCP literature, which was never intended. Prominent school counseling researchers such as Norman Gysbers and Robert Myrick, both seminal in the development of the ASCA National Model, comprehend which program evaluation findings are possible and which are not. It does not make scientific sense for them to build into the model a direct causal assumption (i.e., well-implemented programs in and of themselves will produce higher test scores). This brings me to my second point. There is added benefit in using a variety of research designs and statistical procedures in large- and small-scale CSCP research.

RESEARCH METHODOLOGIES IN CSCP STUDIES

As alluded to earlier, Brown and Trusty (2005) have advocated for designs in school counseling program evaluation research that can establish causality, such as experimental and quasi-experimental approaches. Even though these designs are, in principle, ideal, they are not generally realistic and feasible either by district evaluators or by school-based practitioners. Researchers have already shown that large-scale program evaluations in education using gross performance indicators such as standardized test scores are inherently limited in detecting small but significant and practical group differences (e.g., Goldstein, 2001). Sharply put, no matter what research design is deployed in school counseling studies, it is virtually impossible to make airtight causal statements about the direct effects of any small-scale (proximal) educational innovation let alone systemic programs such as CSCPs (Ellis, 2001; Spirtes, Glymour, & Scheines, 2001). There are too many confounding variables within natural settings to account for without seriously sacrificing the internal, external, and/or ecological validity of the study's findings (Poggenpoel & Myburgh, 2003; Stichter, Clarke, & Dunlap, 2004). Other research designs and statistical methods are therefore needed both at the distal and proximal levels.

Leading counseling researchers such as Heppner, Kivlighan, and Wampold (1999) have recommended for programmatic research agendas to stress paradigmatic diversity to generate the counseling profession's knowledge base. Marzano's (2003) suggestion is helpful in this regard, positing that educational researchers, including school counseling program evaluators, target their investigations at the school, teacher, and student levels, whereas practitioners (e.g., school counselors) can focus their action research on the latter two levels. This is exactly what researchers (e.g., Lapan and his colleagues) did in the studies reviewed by Brown and Trusty and what was, for example, Brigman and Campbell's (2003) research strategy. Although Brown and Trusty focus primarily on conducting efficacy research on proximal interventions (e.g., the effects of group counseling or large-group guidance on various performance indicators) at Marzano's teacher and student levels, they also wisely advocate for distal program assessments, looking at, for instance, how school counselors influence school and classroom climates.

Experimental and quasi-experimental designs should not be the only recommended research methodologies used to evaluate the programmatic or the more localized influence of school counselor-related activities on student achievement. Multiple research (e.g., ex post facto, prediction, longitudinal, time series, qualitative) and statistical approaches are needed (Borders & Drury, 1992; Gysbers & Henderson, 2005; Heppner ct al., 1999; Lapan, 2005; Whiston, 2003). Sophisticated statistical procedures, for instance, using inferred causal modeling (e.g., path analysis and structural equation modeling) are realistic, feasible, and very powerful methods to deploy (Byrne, 2001; Pearl, 2000) in school settings. Other multivariate statistical approaches (e.g., discriminant analysis, hierarchical linear modeling) also can be used effectively to explain a significant proportion of variance in student achievement outcomes. To reiterate, CSCP researchers can still make causal inferences using the findings from nonexperimental studies, but these inferences require researchers to extrapolate from large sample sizes as well as to make explicit the assumptions they used to justify interpreting a parameter estimate as an estimated causal effect (Holland, 2004).

CONCLUDING THOUGHTS--THE CONTINUING NEED FOR SYSTEMS THINKING AND COLLABORATION

Because Brown and Trusty only touch upon the following themes in their article, I underscore them here. The sine qua non of any useful program evaluation is the development and implementation of evaluation schemes that measure clearly defined outcomes within targeted subsystems (Bronfrebrenner & Morris, 1998; Lapan, 2005). Educational and school counseling researchers (e.g., Caron & McLaughlin, 2002; Friend & Cook, 2003; Keys & Green, 2005; Taylor & Adelman, 2000; Whiston, 2003) as well as the ASCA National Model's (2003b) framers emphasized the value of research partnerships as a way to share accountability for student progress in a variety of developmental domains (e.g., career, education, personal-social). Thus, in my view, effective evaluation teams composed of skilled researchers and school-level practitioners using a variety of salient research methods should enhance CSCPs' ability to document over time positive academic development at the program, school, classroom/teacher, and student levels of analysis. Despite the fact that direct causal links are not realistic possibilities in CSCP research, given fairly strict adherence to certain methodological caveats, causal inferences can still be tentatively offered.

References

American School Counselor Association. (2003a). The ASCA national model: Executive summary. Retrieved November 12, 2004, from http://www.schoolcounselor.org/content.asp?contentid=134

American School Counselor Association. (2003b). The ASCA national model: A framework for school counseling programs. Alexandria, VA: Author.

American School Counselor Association. (2004a). Ethical standards for school counselors. Retrieved November 20, 2004, from http://www.schoolcounselor.org/content.asp?contentid=173

American School Counselor Association. (2004b). The role of the professional school counselor. Retrieved November 15, 2004, from http://www.schoolcounselor.org/content.asp?contentid=240

Borders, L. D., & Drury, S. M. (1992). Comprehensive school counseling programs: A review for policymakers and practitioners. Journal of Counseling & Development, 70, 487-498.

Brigman, G., & Campbell, C. (2003). Helping students improve academic achievement and school success behavior. Professional School Counseling, 7, 91-98.

Bronfrebrenner, U., & Morris, R A. (1998).The ecology of developmental processes. In R. M. Lerner (Ed.), Handbook of child psychology (5th ed., Vol. 1, pp. 993-1028). New York: Wiley.

Brown, D., & Trusty, J. (2005). Organizing and leading comprehensive school counseling programs. Pacific Grove, CA: Brooks/Cole.

Byrne, B. M. (2001). Structural equation modeling with AMOS. Mahwah, NJ: Erlbaum.

Caron, E. A., & McLaughlin, M.J. (2002). Indicators of Beacons of Excellence Schools: What do they tell us about collaborative practices? Journal of Educational & Psychological Consultation, 13, 285-313.

Dahir, C., & Stone, C. (2003). Accountability: A M.E.A.S.U.R.E. of the impact school counselors have on student achievement. Professional School Counseling, 6, 214-221.

Ellis, A. K. (2001). Research on educational innovations (3rd ed.). Larchmont, NY: Eye on Education.

Friend, M., & Cook, L. (2003). Interactions: Collaboration skills for school professionals. Boston: Allyn and Bacon.

Goldstein, H. (2001). Using pupil performance data for judging schools and teachers: Scope and limitations. British Educational Research Journal, 27, 433-443.

Gysbers, N. C. (2003).The center of education (Preface). In The ASCA national model: A framework for school counseling programs (pp. 4-5). Alexandria, VA: America n School Counselor Association.

Gysbers, N. C., & Henderson, P. (2005). Designing, implementing, and managing a comprehensive school guidance and counseling program. In C. A. Sink (Ed.), Contemporary school counseling: Theory, research, and practice (pp. 151-183). Boston: Houghton Mifflin.

Heppner, P. P., Kivlighan, D. M., Jr., & Wampold, D. E. (1999). Research design in counseling (2nd ed.). New York: Wadsworth.

Holland, R W. (2004, April). Evidence for causal inference in education research. Paper presented at the AERA Presidential Invited Session on Inference, Evidence and Scientific Research, American Educational Research Association, San Diego, CA. Retrieved November 29, 2004, from http://www.ets.org/research/dload/AERA_2004-Holland.pdf

Keys, S., & Green, A. (2005). Enhancing developmental school counseling programs through collaboration. In C. A. Sink (Ed.), Contemporary school counseling: Theory, research, and practice (pp. 361-389). Boston: Houghton Mifflin.

Lapan, R. (2005). Evaluating school counseling programs. In C. A. Sink (Ed.), Contemporary school counseling: Theory, research, and practice (pp. 257-293). Boston: Houghton Mifflin.

Marzano, R. J. (2003). What works in schools: Translating research into action. Alexandria, VA: Association for Supervision and Curriculum Development.

Musheno, S., & Talbert, M. (2002).The transformed school counselor in action. Theory Into Practice, 41, 186-191.

Paisley, P. O., & McMahon, H. G. (2001). School counseling for the twenty-first century: Challenges and opportunities. Professional School Counseling, 5, 106-115.

Pearl, J. (2000). Causality: Models, reasoning, and inference. New York: Cambridge University Press.

Poggenpoel, M., & Myburgh, C. (2003).The researcher as research instrument in educational research: A possible threat to trustworthiness? (A:research_instrument). Education, 124, 418-421.

Spirtes, P., Glymour, C., & Scheines, R. (2001). Causation, prediction, and search (2nd ed.). Cambridge, MA: MIT Press.

Stichter, J., Clarke, S., & Dunlap, G. (2004). An analysis of trends regarding proactive and ecologically valid interventions in applied research. Education and Treatment of Children, 27, 86-104.

Taylor, L., & Adelman, H. S. (2000). Connecting schools, families, and communities. Professional School Counseling, 3, 298-308.

Whiston, S. (2003). Outcomes research and school counseling services. In B.T. Erford (Ed.), Transforming the school counseling profession (pp. 435-447). Upper Saddle River, NJ: Prentice Hall.

Christopher A. Sink, Ph.D., NCC, LMHC, is a professor and chair, School Counseling and Psychology, School of Education, Seattle Pacific University, Seattle, WA. E-mail: [e-mail]