Who is smarter, CTE or other students? A five-year high-stakes test score comparison answers the question.

THE QUESTION SHOULD ACTUALLY BE, "IS A RAW score comparison a fair and accurate measure between the two groups?"

The answer is: No, career and technical education (CTE) students scored lower than other students in four out of five years during this study on the high-stakes tests. Yet factors other than curriculum choice were found to have a significant influence on the high-stakes test scores. In fact, no statistical difference was found between the two groups. Understanding and articulating this phenomenon to policymakers, administrators and the public is essential for all career and technical educators.

High-stakes, test-based reform used to be driven by state-level mandates because of the political appetite for rapid, quantifiable results even though the federal government passed the Elementary and Secondary Education Act (ESEA) in 1965. However, in 1983, "A Nation at Risk" was introduced as a reform effort for the public school system. It was soon followed by yet another effort of reform in 1987 with America 2000. This was not the end of the reform assault on education. The Clinton Administration introduced Goals 2000, which marked the third major or national attempt at reform of the nation's educational system within the last two decades. The most recent ESEA is known as the No Child Left Behind (NCLB) Act signed by President Bush in 2001. This Act asked states to set standards for student performance and teacher quality. High-stakes testing continues to play a prominent role in this legislation (U.S. Department of Education, 2004).

Assessment has become a part of life, whether it is in the classroom or on the job. Most state departments of education use standardized tests to determine the progress of students. Standardized tests should assess higher order reasoning skills and academic growth over time, but do they (Sanders and Horn, 1995)?

Comparison is the main issue with standardized testing. It provides a way of comparing groups, states, schools and teachers. These tests are often considered the most visible indicator of education successes and failures. Therefore, it is critical that the assessment and curriculum be aligned so as to give the greatest benefit to fairness (Nielson, 1985).

Problem Statement

The issue at hand is what type of student achieves at a higher level? Do students with strong academic coursework achieve at a higher level than those students who have a CTE background? It seems that there is a concern about the progress of CTE students and how they perform on state mandated achievement tests versus students who have minimal or no CTE coursework. There are many other influences that may affect student scores such as ethnicity, socioeconomic status (SES), gender, parents' education, Individual Vocational Education Plans (IVEP), and learning styles.

Purpose

The purpose of this research project was to compare CTE students and non-CTE students on the high-stakes tests while controlling for extraneous variables such as learning styles, special populations, gender, race and ethnicity.

Background

High-stakes testing is a concept that began in 1983 as a direct result of "A Nation at Risk'--a report released by the National Commission on Education. According to Amerin and Berliner (2002), this report proposed an end to minimum competency testing and called for a movement that would raise the nation's standards of achievement. This report led legislators to require the development of educational standards. As a result, state policymakers created educational standards (Amerin and Berliner, 2002).

Multiple measures of assessment are typically used to yield valid results (Liu, 1997). At the intermediate level the standardized test is used to compare students with their peers from similar learning settings (Cooper, 1995). Secondary CTE continues to suffer from a negative image among students, parents, educators and policymakers. It is plagued by the negative stereotype that CTE is only for non-college bound, potential dropouts, and other students with various special needs. However, CTE programs assist students in developing and improving their problem-solving skills, which in turn aids in the process of critical thinking (Cano, 1993). These students may actually be at an enhanced level when talking about assessment. They are often skilled at taking various forms of assessment as compared to non-CTE students who may only be exposed to one or two different assessment styles.

Standardized tests are known to vary in their ability to fairly assess students' knowledge (Sanders and Horn, 1995). High-stakes tests are often the most visible indicator of successes and failures for students, parents as well as the community (Nielson, 1985). Tests have come to define schools' priorities. Schools are focusing on core subjects and are forced to eliminate CTE and fine arts programs. These programs may not be directly correlated to the standards being assessed by the state mandated tests, but they do serve a purpose. Students learn more efficiently when they are actively engaged in a project or task. CTE allows for various learning styles and exploration. A growing body of evidence indicates that learning styles are important for both understanding and managing the teaching-learning process.

Schools across the nation are becoming mass "test-prep" facilities. Schools under intense pressure to show better test results have allowed those tests to cannibalize the curriculum (Kohn, 2001). The curriculum becomes narrow as school district policies make it clear that what is to be tested is what is to be taught (Eisner, 2001).

NCLB is being implemented in all 50 states. Many states now offer incentives for high scores or for great improvement shown on the tests. Test scores play a critical part in letting the public know about schools', districts' and states' accomplishments. Real estate companies use test scores to determine neighborhood quality, which in turn sets the market value of properties. Properties have been increased by as much as $9,000 based upon the test scores of the local schools (Amerin and Berliner, 2002). Some say that high expectations for students improve academic achievement and that students excel when they are challenged. Most departments of education agree with this statement and have hence created academic standards.

According to most academic standards (1998), students should know and be able to exhibit certain levels of achievement at various ages in their academic careers. In one state, these competencies are defined as follows: readiness (kindergarten); foundations (grade one-three); essentials (grade four-eight); proficiency (grade nine-12); and then distinctions (with honors). Provided within each standard, performance objectives have been developed which indicate whether students possess the knowledge and/or skills they should have by that grade level.

Procedures

The two comparison groups (Campbell and Stanley, 1963) were the CTE and the non-CTE students. The 2,500 participants of this study were high school students representing five school districts from three different geographical regions. The groups were compared on their high-stakes test scores for five years. Factors tested to determine if they influenced the test scores included: curriculum choice; CTE concentrator level or non-CTE concentrator; gender; race/ethnicity; special populations (handicapped, limited English proficiency, economically disadvantaged, academically disadvantaged, and single parent); and learning styles (visual, auditory and kinesthetic).

The high-stakes test used in this study--Arizona Instrument to Measure Standards--was developed for the Arizona Department of Education. The second instrument used in this study was a learning styles assessment that was administered to students in all five districts participating in the study. Every student involved in the study was asked to complete the learning styles assessment that was then calculated to arrive at the various learning styles of the targeted students. The remaining data (curriculum choice, gender, race/ethnicity and WED were collected from the Arizona education department's records.

Findings

White students were the most prevalent group. Gender was almost equally divided between female and male. Students receiving IVEP services scored significantly higher on the high-stakes test than those students who were eligible for services, but did not receive them. The important variable of interest--if a student is a CTE concentrator-was not significantly associated with either higher or lower test scores.

All five special population areas (handicapped, limited English proficiency, economically disadvantaged, academically disadvantaged and being a single parent) were significantly associated with lower test scores and were predominantly found in the CTE population. Higher visual learner (learn by seeing) and auditory learner (learn by hearing) scores were significantly associated with higher test scores and were predominantly found with other students. Higher kinesthetic learner scores were significantly associated with lower test scores and were predominantly found with CTE students. Black, Hispanic or other males were associated with lower test scores. Hispanic females were associated with lower test scores. The results were found for all five years even though in 2003 CTE students actually had higher raw scores.

Conclusions, Implications and Recommendations

Grouping students according to their special population status affects the comparison because a higher proportion of students who received special population services and who learn kinesthetically are located in CTE programs as compared to academic programs. Therefore, after controlling for the extraneous influences, no difference was found between the two groups five years in a row with approximately 2,500 students studied each year. The conclusions are simple: Career and technical education students, for the most part, will always do worse on high-stakes test raw score comparisons because CTE attracts students whose learning styles and other characteristics do not lend themselves to scoring well on high-stakes tests. When the appropriate extraneous variables are built into the equation and controlled, there usually is no difference between CTE and other students on standardized test scores. The raw score comparisons are inappropriate because the groups are different.

The differences in scores between the groups can be attributed to the effects of the extraneous variables and not because of curriculum choice. The implications for educators are important--if extraneous effects are not understood and controlled, then CTE will not have a very positive future. Therefore, the recommendations are that CTE administrators and teachers must understand the problems associated with raw score comparisons on standardized tests. CTE state leaders meanwhile, must utilize this type of information in CTE promotional campaigns.

References

Amerin, A.L. & Berliner, D.C. (2002, March 28). "High-stakes testing, uncertainty, and student learning." Education Policy Analysis Archives, 10(18).

Brown, S. (1996). Assessment (online) www.lgu.ac.uk/deliberations/ assessment/invite.htm.

Campbell, D. T., & Stanley, J. C. (1963). "Experimental and Quasi-experimental Designs for Research." Chicago: Rand McNally.

Cano, J. (1993). "An Assessment of the Level of Cognitive Performance and Critical Thinking Ability of Selected Agricultural Education Students." Journal of Agricultural Education.

Cooper, E. (1995). "Using Standardized Test Results To Assess Student Learning." ERIC/ AE Digest. Boaz, Alabama.

Department of Education. "2004 Standards-based Teaching and Learning Academic K-12 Standards."

Jewell, L. (1989). Opinions of School Administrators Concerning the Purpose,

Community Acceptance and Occupational Placement as a Basis for Justification of Vocational Agriculture Programs." Journal of Agricultural Education.

Kohn, A. (2001). "Fighting the Tests: A Practical Guide to Rescuing Our Schools." Phi Delta Kappan, Vol. 82 Issue 5.

Kossan, Pat. (2005). "3rd time may be charm on AIMS." The Republic. Retrieved March 22, 2005 from www.azcentral. com.

Liu, K. (1997). "Educators' Responses To LEP Students' Participation in the 1997 Basic Standards Testing." University of Minnesota.

Nielsen, N. (1985). Aligning Assessment with Learning Goals. American Association for the Advancement of Science.

Raven, M. (1990). "Influence of Enrollment in Vocational Agriculture on Admissions to a College of Agriculture." Journal of Agricultural Education.

Sacks, P. (2000). Standardized Minds: The High Price of America's Testing Culture and What We Can Do To Change It. Perseus Books, Cambridge, Mass.

Sanders, W. & Horn, S. (1995). "Educational Assessment Reassessed: The Usefulness of Standardized and Alternative Measures of Student Achievement as Indicators for the Assessment of Educational outcomes." Education Policy Analysis Archives Vol. 3 Number 6 March 1995.

State Department of Education, (1998). "Academic Standards: Student Achievement Program." State Department of Education. (ERIC Document Reproduction Services No. ED456164).

U.S Department of Education, Office of the Secretary, Office of Public Affairs (2004). "A Guide to Education and No Child Left Behind," Washington, D.C., 2004.

Jack Elliot is professor and heed of agricultural education et the University of Arizona. He can be reached at [e-mail].