Researching CTE student success: a new conceptual framework.

FOR CAREER AND TECHNICAL EDUCATION (CTE) practitioners, researchers and policymakers to see themselves as central players in the emerging consensus regarding the changing future of public education, they must all come together as a new frontline to establish evidence-based accountability systems that make the case for measuring CTE student success. These evidence-based accountability systems must be built using common measures, methodologies and metrics.

A starting point for defining CTE student success can be described in terms of three elements: engagement--defined as attending, focusing and specializing in coursework and work-based learning within programmatic career pathways and programs of study; achievement--defined as academic performance, skill development, and completing (graduating) school or college; and transition--defined either as high school graduates moving on to postsecondary education without the need for remediation or as managing the learning swirl that is taking place between education and the workplace.

Research shows that higher engagement levels ensure a reduction in the probability of dropping out of school, which in turn does not limit postsecondary education (Castellano at. al., 2002).

On the other hand, exclusive concentration on CTE in high school may lead to a reduction in college enrollment (De Luca et. al., 2006). Students achieve better (higher grade point averages, increased test scores, higher graduation rates, increased college and university enrollments) when there is an increase in academic course taking of CTE students but within a curriculum integration framework--and when CTE students are placed in smaller learning communities (such as career academies) that have well-defined career pathways (Castellano at. al., 2004; Stone and Aliaga, 2003).

Much of the more recent research debate on transitions has centered on college and work readiness (ACT, 2004; Byrd, 2005; Dounay, 2006). The impetus to target college readiness comes from the No Child Left Behind (NCLB) law, with its exclusive focus on increased academic requirements in math, science and language arts. Some have argued that high school graduates, in addition to increased academics, need to have a concentration in CTE. NCLB requires more rigorous academics, but what might be occurring in high schools is the replacement of CTE with remedial courses. In other words, high school graduates transition to postsecondary with neither rigorous academics nor a proper concentration in CTE (Elliot and Deimler, 2007). Thus, to avoid remediation in high school and in college, the two must partner to jointly define college readiness standards and ensure that college readiness includes a concentration in CTE (SREB, 2006).

Without increased secondary and post-secondary collaboration, there will be no guarantee that high school students will not have to repeat remedial courses in college. It will also not lead to closing the achievement gap nor enable effective transitions for all students, not just CTE students, to the postsecondary system and work.

A New Conceptual Framework for CTE Research

To develop a conceptual framework for CTE research, the following needs to be taken into account. First, a precise definition of who a CTE student is, and that definition should clearly indicate the boundaries of his or her experiences. Second, develop a standard data system that will keep track of a CTE student's educational progression across well-defined career pathways that span grades nine-14. Third, at the different stop-out points CTE students encounter along a career pathway, measures must be constructed precisely to indicate the level and extent of success. Last but not least, the back and forth transitions between education and employment (the learning swirl) of many CTE students must be clearly taken into account.

Taking these four elements into consideration, valid and reliable CTE student success measures would then cover the gamut, from high school graduation through college readiness, technical skill proficiency and postsecondary credentials, with employment success and value-added wage earnings in a self-determined career pathway.

Figure 1 shows the choice high school and college students have of three different pathways: general, academic or CTE (see inside the circled area in Figure 1). As Figure 1 shows, both the NCLB model (exclusively academic courses and programs) and the pure vocational model (exclusively CTE courses and programs) can be incorporated into a third: the contextual learning model, which is explained as follows.

[FIGURE 1 OMITTED]

A general pathway usually means students do not focus on either an academic program or a CTE program. A student chooses an academic pathway by concentrating in an academic area, even though he or she might be introduced to CTE through some course taking. A CTE pathway is one in which students combine CTE course taking with academic course taking within a CTE program of study. Assessing CTE student success at various self-determined stop-out points along a chosen career path then becomes a matter of comparing measures among students in the three different pathways. A variant of this conceptual framework has been put into practice in Minnesota.

Methodology

With one primary source of all student information, called the Integrated Student Record System (ISRS), the Minnesota State Colleges and Universities system (www. mnscu.edu) is able to access, summarize and report a wide variety of information covering demographics, performance and other student characteristics. Additionally, using information already available in ISRS, early on under the 1998 Carl D. Perkins Act (Perkins III), Minnesota decided to construct a unified database and collect all specific postsecondary CTE-related information for two-year college students within the Minnesota State Colleges and Universities system in one place.

Known as the Perkins Brio database, the primary purpose so far has been to meet the Perkins III accountability requirements for compliance. However, with an increased focus on accountability under the 2006 Carl D. Perkins Act (Perkins IV), the Perkins Brio database continues to be made more integral and dynamic and to be positioned more as an evidence-based accountability system.

As shown in Figure 2, the ability to link certain information in ISRS to the Perkins Brio database has led to the construction of a research database, thereby enabling the development of several new avenues of research on CTE student success in Minnesota. Before discussing the results and implications that emerged from the three research studies, a note on the methodology used might be in order.

[FIGURE 2 OMITTED]

While the particular methodology is specific to each research study, it generally follows a common procedure that involves the following steps. One, a cohort of students, representing each pathway variant (described in Figure 1), is collected from a database as depicted in Figure 2. The sample is divided into two categories: those focusing on CTE programs, clusters and awards, all of which are potentially eligible for Perkins funding; and, those focusing on non-CTE programs, clusters and awards (usually the associate of arts degree and some associate of science degrees), which are not eligible for Perkins funding. Two, the relevant and pertinent independent variables are used either in their original form or redefined to obtain a more precise representation of these explanatory variables. Three, the critical variables of interest are pinpointed and refined into dependent variables, and together with the independent variables, made ready for analysis. Four, the statistical techniques and method of analysis are finally chosen, from which the results are derived.

While it is true that the methodology used in the research studies is not based on a randomized experimental research procedure, nevertheless, having a large sample data pool (in the tens of thousands) to choose from tends to mitigate any issues regarding sample validity and reliability.

What Does CTE Student Success Look Like?

Using a common database and analytical framework, over the past three years, this author has conducted research and data analysis on several CTE-related topics. Results from these research studies help shape the development of the conceptual framework outlined above for researching and analyzing CTE students' success in terms of their engagement, achievement and transitions.

Among the topics explored are (a) impact of dual enrollment programs on college performance and graduation of CTE students; (b) the timing and sequencing of math course taking in the credential progression and completion of a CTE student; and (c) the relationship between different stop-out points--enrollment, credential progression and graduation--of CTE students and the value added to wage earnings.

Dual Enrollment and Student Success in College. When high school students enroll in the Postsecondary Enrollment Options (PSEO, the name of the dual enrollment program in Minnesota), a combination of technical and academic courses in a dual enrollment program generally leads to a higher probability of receiving a postsecondary credential than if the PSEO student took either exclusively academic or exclusively technical courses. Also, college students with prior PSEO experience tend to have higher mean GPAs than those without the experience.

The GPA differences appear to be early in the students' college careers, giving them a "head start." But these early gains dissipate once the student concentrates on CTE and moves closer toward receiving a degree. Nevertheless, the head start is critical if it means that when high school students encounter the right combination of academic and technical courses, the early success in college goes a long way in ensuring that all students, but especially CTE students, continue toward completing their college program of study and leave with a postsecondary credential.

Math Course Taking, Under-Preparedness in College Math and Student Success.

Having success in math course taking tends to improve overall GPAs for both CTE and associate degree-seeking students. For CTE students to raise their overall GPAs, they must take their first nondevelopmental math course in the year following enrollment; for associate degree students, it makes no difference to their overall GPAs when they take their nondevelopmental math courses.

Receiving an award is not affected by whether a CTE or an associate degree student passes his or her first nondevelopment math course, but graduation is ensured if they take math sometime during their program, preferably in the year following enrollment. For associate degree students, the odds for graduating improve if they delay math taking; for CTE students, the window of opportunity is much narrower--when CTE students take math earlier in their program, they are assured higher odds of graduation.

CTE Educational Programming and Value Added in Wage Earnings. Value added in wage earnings is the highest when CTE students convert their intent to receive any type of degree to an award. However, for certificates, it does not matter if they actually complete the credential, and value added in wage earnings rises regardless. With associate degrees, it is clear that students deciding to "major" in this credential must complete the program to see substantial wage gains. Hence, for colleges, starting certificates programs and marketing them as a first step toward a solid CTE career path leading to a wage-enhancing credential would be a worthwhile strategy for improving enrollment in CTE. On the other hand, for degree programs with higher credit levels, the institutional focus should be on retention strategies that move students toward completing associate degrees.

Leaving before receiving a credential does not add value to wage earnings, but it is much less severe for men than it is for women. The same can be said for whites and minority students--less severe for whites than for minorities. However, it is much more crucial for minorities to "major" in some credential, since the value added to wage earnings rises considerably when they do complete the credential, particularly for shorter ones like certificates rather than longer ones like associate degrees. Thus, regardless of whether they finish or not, minority students must be directed toward certificate programs, since wage gains for them are rather substantial.

The value added to wage earnings gets increasingly smaller when students enter college at a later age; in fact, the gain turns negative for the oldest age groups. Hence, colleges might think of developing age-group-specific retention and completion strategies, such as focusing on certificates and diplomas. Nevertheless, options might be limited, since such students returning to college often come from occupations and industries that have become victim to the changing global dynamics of the "flat" world.

A broad implication derived from the results appears to be that CTE students must be given options such as dual enrollment and math course taking beyond remedial early enough. By taking advantage of these options, CTE students have a greater chance for higher performance levels and greater success of graduating from college. Moreover, by following a definite credential progression and completion path--particularly for women and minorities--within a well-defined career pathway, the value added to wage earnings rises substantially. More generally, the research results from Minnesota highlight the need for a common conceptual framework if CTE student success is to be set against that of other non-CTE students using a uniform metric of measurement and analysis.

References

ACT, Crisis at the Core: Preparing All Students for College and Work, 2004.

Byrd, Kathleen L. and MacDonald, Ginger; "Defining College Readiness from the Inside Out: First-Generation College Student Perspectives," Community College Review, Fall 2005 v.33.

Castellano, M., Stringfield, S., & Stone, J. R., III (2002); Helping Disadvantaged Youth Succeed in School: Second-Year Findings from a Longitudinal Study of CTE-Based Whole-School Reforms; St. Paul, Minn.: National Research Center for Career and Technical Education, www. nccte.org.

Castellano, M., Stone, J. R., III, Stringfield, S., Farley, E. N., & Wayman, J. C. (2004); The Effect of CTE-Enhanced Whole-School Reform on Student Course Taking and Performance in English and Science; St. Paul, Minn.: National Research Center for Career and Technical Education.

DeLuca, S., Plank, S., & Estacion, A. (2006); Does Career and Technical Education Affect College Enrollment?; St. Paul, Minn.: National Research Center for Career and Technical Education.

Dounay, Jennifer; High School--Transition to Postsecondary Education Embedding College Readiness Indicators in High School Curriculum and Assessments, Education Commission of the States Policy Brief, April, 2006.

Elliott, Jack and Deimler, Buddy; "The Premier Education Delivery System," Techniques, January 2007.

Kotamraju, Pradeep, "The Minnesota Postsecondary Enrollment Options Program: Does Participation in Dual Enrollment Programs Help High School Students Attain Career and Technical Education Majors and Degrees in College?" Annual Meeting of the Association of Institutional Research, San Diego, California, May 2005.

Kotamraju, Pradeep; "Getting Timing and Sequencing of Math Course Taking within Specified Career Pathways Right: A Possible Answer to the Under-preparedness in Math of Career and Technical Education (CTE) Students," Annual Meeting of the Association of Institutional Research, Chicago, Illinois, May 2006a.

Kotamraju, Pradeep, "Using Graduation as an Education Accountability Measure: Does it Tell us the Complete Picture When it Comes to Workforce Development?" Annual Meeting of the National Council of Workforce Education, Albuquerque, New Mexico, October 2006b.

Stone, J. R., III, & Aliaga, O. A. (2003); Career and Technical Education, Career Pathways, and Work-Based Learning: Changes in Participation 1997-1999; St. Paul, Minn.: National Research Center for Career and Technical Education.

Southern Regional Education Board, Getting Students Ready for College and Careers, 2006, www.sreb.org.

Pradaap Katamraju is system director for Perkins federal grants with the Office of the Chancellor at Minnesota State Colleges and Universities in St. Paul, Minnesota. He can be contacted at pradeep.kotemraju@ so.mnscu.edu.