Do intervention programs assist students to succeed in college? A multilevel longitudinal study.
It is reported that about one fourth of students dropped college after their first year (Mallinckrodt & Sedlacek, 1987; Tinto, 1993; Tinto, Russo, & Kadel, 1994), and about 25% of college graduates got their bachelor degrees from the first college they attended (National Center for Education Statistics, 2004). Retention as well as academic performance is critical to students' success in college; therefore, they are important issues in college administration.
Retention is a complex issue involving many different factors. Whether a student departs from an institution is largely a result of the extent to which the student becomes academically and socially connected with the institution. As Tinto's (1975) model indicates, as students are integrated into and become more interdependent with both academic and social elements of a university, the probability that the student will leave the university declines. Astin (1975) also found that involvement was critical to a student's decision to persist or drop out school. In other words, involvement with faculty and student peer groups encourages participation in social and intellectual life of a college and, therefore, helps learning and persistence in college (Astin, 1993; Berger, 1999; Campbell & Campbell, 1997; DeBerard, Spielmans, & Julka, 2004; Nagda, Gregerman, Jonides, von Hippel, & Lerner, 1998; Tinto, 1993).
Other factors that may also affect retention and academic performance include institutional type (Chapman & Pascarella, 1983), motivations for attending college (Allen, 1999; Stage, 1989), financial aid (Cabrera, Nora, & Castaneda, 1992; Glynn, Saner, & Miller, 2003; Sandler, 2000; Wetzel, O'Toole, & Peterson, 1999), fulfillment of expectations for college (Braxton, Vesper, & Hosler, 1995; Glynn et al., 2003), sense of community in residence halls (Brower, 1992), self-efficacy (Peterson, 1993), attitudes (Fishbein & Ajzen, 1975; Glynn et al., 2003), and maladaptive coping strategies (DeBerard et al., 2004). In addition, previous research results show that interactions of those factors with students' characteristics, e.g., demographics and college preparedness, play an important role on their success in college.
In the literature, most research on students' success in college has used student enrollment data to explore factors affecting students' success in college. Such research has provided much valuable information to college administrators, faculty and staff. As a result, many universities have setup programs based on the theories in the field and the results of research trying to improve students' academic achievement and prevent students from attrition. However, research on intervention program effects somewhat falls behind. Such research would make an important contribute to the literature.
This present study is an effort to examine the intervention programs conducted in a Midwest urban university. In an effort to manage the attrition problem and improve students' academic performance, the university has initiated nearly 100 intervention programs, from 2001 to 2003, with the Success Challenge grant funded by the state. The Success Challenge grant has two components: 1) challenging university campuses to enable at-risk students successfully to earn baccalaureate degrees; and 2) challenging university campuses to enable baccalaureate seeking students to complete their degrees in a timely fashion, typically four years. In theory, these programs were set up based on Tinto's (1975) Student Integration Model and Astin's (1975) Theory of Involvement. Most of the interventions programs were designed to promote student-to-student interaction, faculty-to-student interaction, student involvement, academic engagement, and academic assistance. They can be roughly categorized into six different program types based on the types of services they provide. The categories consist of advising, academic help, first-year experience (FYE), social integration, general orientation, and financial aid. The Financial Aid category was dropped in the present study due to the small number of students in that group.
Murtaugh, Burns, & Schuster, (1999) stated that freshman orientation may be effective to reduce the risk of dropping out. It is not clear whether the other types of intervention are effective or not, and how long the effect lasts. It is also a great concern to the stakeholders and the administrators about which programs work, how they work, and which one works better, and to whom. The purpose of this present study is to try to answer these questions by examining the effects of the intervention programs on retention and students' academic performance which is measured by college cumulative GPA across 3 years, interacting with students' characteristics, using multilevel longitudinal modeling.
Setting and Participants
1305 First-Time-Full-Time (FTFT) students who voluntarily participated in one of 20 Success Challenge intervention programs at the beginning of the fall quarter of 2000 at a large Midwest urban university were included in the study. The mean age was 18.62, with SD = .56. The sample is made up 46.7% female. The sample has 83.2% Caucasian, 11.3% African-American, and 5.5% other race. The average high school GPA was 2.91 (SD = .64). In addition, 18% of the participants were from high selective colleges, 11% moderate selective, 39% liberal selective and 32% not selective. Table 1 shows the descriptive statistics by program type.
Success Challenge Programs
Advising programs. Advising programs included programs such as Central Advising, the Pre-Professional Advising Center, and Career Navigator. Central Advising consists of satellite advising centers staffed by a team of professional academic advisors. The Pre-Professional Advising Center provides general advising to students who have an interest in exploring pre-professional majors. The Career Navigator series is a six-stage program designed to help freshmen and sophomores to explore potential majors and careers.
Academic help programs. Academic help programs included Tutorial Services, Co-op Calculus, and the Engineering Aerospace Collaborative. Tutorial services serve to improve students' grades in specific courses and empower the students as active, independent learners. Co-op calculus is designed to help freshmen succeed in the introductory calculus sequence. The Engineering Aerospace Collaborative selected underrepresented pre-engineering students to take part in activities such as a summer bridge program, introductory engineering workshops, and cooperative learning courses in math and physics.
First Year Experience programs. FYE courses are designed to incorporate a core set of common classroom experiences focusing on college survival skills, transition issues, and career and personal development. Some of the FYE courses include FYE Librarian and Orientation to Learning. The purpose of FYE Librarian is to instruct first year students on how to use the many research resources and services available at the university to further develop their research skills. Orientation to Learning is a course requirement for all incoming freshmen. It is designed to help first year students understand the many challenges of college life and to introduce them to the many resources available to them on campus.
Social integration programs. Social integration programs served to increase both student-to-student and faculty-to-student interactions. The programs under this category included Learning Communities and Faculty/Student Activities. Learning Communities is a course structure created by students registering for a block of classes together. Students in learning communities form supportive peer groups, which help them to become both academically and socially connected. The Faculty/Student Activities program hosts a number of events which bring students and faculty together outside of the classroom.
General orientation programs. Orientation programs provide students with a description of program offerings, college expectations, information about assistance and services for examining interests and abilities, encouragement to establish working relationships with faculty, information about services that help with adjustment to college, and financial aid information.
Data Source and Measures
The data were retrieved from the university enrollment database and the Success Challenge Program records. The current study examined two outcome measures: (a) the retention rates for the three fall quarters of the academic years of 2001-2002, 2002-2003, and 2003-2004; and (b) the college cumulative GPA for the three academic years of 2000-2001, 2001-2002, and 2002-2003. The program categories of Advising, First-year experience, Social Integration, Academic Help, and General Orientation served as the characteristics of the programs based on their service type. The students' characteristics included in the present study were gender, ethnicity, high school GPA, and college selectivity. Students' age was not modeled since it was not a significant factor in this present study.
Since the data were nested in nature, that is, the repeated observations (three-year retention rates and college cumulative GPA) were nested within the individuals and the individuals in turn were nested within the programs, three-level hierarchical modeling (HLM; Raudenbush & Bryk, 2001) was conducted for examining the program effects on the retention and the college cumulative GPA. For the retention, a 3-level logistic model was used because the outcome variable was dichotomous. The first level model was about the three-year retention rates, the second level model was about the students, and the third level model was about the programs. The students' characteristics (e.g., gender, ethnicity, and high school GPA) and the program characteristics (e.g., program types and percent female students in the program) were modeled at levels 2 and 3, respectively. For academic performance, we treated the college cumulative GPA as a continuous variable, thus a 3-level regression model was conducted. The first level model was about the three-year college cumulative GPA, the second level model was about the students, and the third level model was about the programs. The time-varying variables (e.g., residence and course load), the students' characteristics (e.g., gender, ethnicity, and high school GPA) and the program characteristics (e.g., program types) were modeled at levels 1, 2 and 3, respectively.
The retention rates across the fall quarters of the three academic years were .67, .54, and .49, with a mean of .57. The three-year cumulative university GPA were 2.33 (SD = .97), 2.75 (SD = .62), and 2.58 (SD = .73), respectively, with a mean of 2.52 (SD = .84).
Program Effects on Retention
An unconditional hierarchical logistic linear model revealed that 22.54% of total variation in the logit of retention was among the programs, which justified the use of a hierarchical linear modeling. This result also indicates that different Success Challenge Programs had different significant effects on retention. In order to find which intervention program and what student characteristics explained a 3-year retention trend, a three-level logistic HLM longitudinal model was conducted.
Table 2 displays the parameter estimates from the multilevel longitudinal analysis for retention. From Table 2, we can see that the major findings are: (a) The academic-help programs significantly ([[gamma].sub.001] = .997, p < .010) increased the retention rates for the first year; (b) The advising ([[gamm].sub.011] = .537, p < .001) and social integration ([[gamma].sub.012] = .487, p < .001) programs significantly helped students who were in higher selective colleges return to school after the first year; (c) Female students more likely ([[gamma].sub.020] = .268, p< .046) returned to school after first year; and (d) Students with higher high school GPA more likely returned to school after first year ([[gamma].sub.030] = .412, p < .005) and also more likely returned to school for the following two years ([[gamma].sub.110] = .171, p < .009).
Program Effects on Academic Performance (GPA)
An unconditional HLM revealed that 17.93% of total variation in the academic performance measured by the college cumulative GPA was among the programs. In other words, for students who participated in different programs would have different GPAs, which may also show that the Success Challenge Programs had significant different effects on the college cumulative GPA. An HLM longitudinal model, controlling for two significant time-varying covariates, cumulative credit hours and residence (dormitory vs. other place), was conducted for exploring what students' characteristics and programs helped the students' GPA.
Table 3 shows the parameter estimates from multilevel longitudinal analysis for GPA.
The main findings are: (a) The general orientation programs significantly ([[gamma].sub.001] = .374, p < .001) helped all students increase GPA for the first year; (b) The social integration programs significantly ([[gamma].sub.011] = .175, p < .001) helped students who were in selective colleges increase their GPAs for the first year; (c) The GPA increasing trend was significantly ([[gamma].sub.101] = -.061, p < .001) slower for students who participated in the general orientation programs than students who participated in the other programs; (d) Female students ([[gamma].sub.020] = .090, p < .006), White students ([[gamma].sub.030] = .306, p < .001), or students with higher high school GPA performed significantly higher in GPA ([[gamma].sub.040] = .532, p< .001) than other students for the first year; and (e) For White students ([[gamma].sub.110] = -.052, p < .001), African-American students ([[gamma].sub.120] = -.078, p < .001), or students with higher high school GPA ([[gamma].sub.130] = -.024, p < .036), the increasing GPA trend across the three years was significantly slower than other students.
This study utilized multilevel longitudinal modeling to explore the effectiveness of the intervention programs on FTFT students' retention and academic achievement measured by college cumulative GPA. The findings of the study imply: 1) early intervention programs assist to retain first year students in college; 2) academic-help programs help participants to return; 3) for those who are better prepared for college, social interactions with faculty, staff, and their peers enhance their returning to school; 4) social integration programs increase GPA for students who are in more selective colleges; and 5) general orientation helps students increase GPA at early stage, but the effect did not necessarily last.
This study confirmed Tinto's (1993) statement that involvement in social and intellectual life of a college helps learning and persistence in college. Other studies (Alikonis, Guo, & Miller, 2005) about the Success Challenge programs show that participation in more than one Success challenge program greatly helped student both in retention and increase of GPA, not only in the first year, but also second and third year.
The results of this study show that general orientation is necessary at the beginning of the college life. School administrators may also need to focus more on social- and academic-help-related specific programs other than just general orientation to help student return to school and increase GPA. For better college prepared students, universities may need programs that promote student social interactions with faculty, staff and their peers to help them return to their schools. For the under prepared students, academic help is more needed. A combination of academic help and social interaction may work better.
It would give us a deeper insight if student's psychological factors and family background were added into the models. If more program characteristics were available, the program effects could be explored more thoroughly. For further study, more program information, such as who ran the program, budgetary information, the length of the program may explain more clearly about the program effects. The students in the current study participated in only one of the programs, but there were almost equal number of students who participated in multiple programs. Thus, the program effects on students' retention rates and academic performance were under estimated for the all FTFT students. Moreover, this study did not intent to draw a causal effect of the intervention programs on students' retention and academic performance for all FTFT students in the population, because the data were retrieved by convenience from the database rather than randomly selected from the population. Nonetheless, this study at least provided an empirical evidence of the effect of the intervention programs on students' retention and academic performance.
Alikonis, C., Guo, S., & Miller, C. (2005, May). Success challenge: Retention efforts at a Midwest urban university. Paper presented at the 45th Forum of the Association of Institutional Research, San Diego, CA.
Allen, D. (1999). Desire to finish college: An empirical link between motivation and persistence. Research in Higher Education, 40, 461-485.
Astin, A. W. (1975). Preventing students from dropping out. San Francisco: Jossey-Bass Publishers.
Astin, A. W. (1993). What matters in college?: Four critical years revisited. San Francisco: Josey-Bass Publishers.
Berger, J., & Milem, J. (1999). The role of student involvement and perceptions of integration in a causal model of student persistence. Research in Higher Education, 40, 641-664.
Braxton, J. M., Vesper, N., & Hosler, D. (1995). Expectations for college and student persistence. Research in Higher Education, 13, 595-612.
Brower, A. (1992). The "Second Half" of student integration: The effects of life task predominance on student persistence. Journal of Higher Education, 63, 441-462.
Cabrera, A. F., Nora, A., & Castaneda, M. B. (1992). The role of finances in the persistence process: A structural model. Research in High Education, 33, 571-593.
Campbell, T. A., & Campbell, D. E. (1997). Faculty/student mentor program: Effects on academic performance and retention. Research in Higher Education. 38, 727-742.
Chapman, D., & Pascarella, E. (1983). Predictors of academic and social integration of college students. Research in Higher Education, 19, 295-322.
DeBerard, M. S., Spielmans, G. I., & Julka, D. L. (2004). Predictors of academic achievement and retention among college freshmen: A longitudinal study. College Student Journal, 38, 66-80.
Fishbein, M., & Ajzen, 1. (1975). Belief attitude, intention and behavior. Reading, MA: Addison-Wesley.
Glynn, J. G., Sauer, P. L. & Miller, T. E. (2003). Signaling student retention with prematriculation data. NASPA Journal, 41, 41-67.
Mallinckrodt, B., & Sedlacek, W. E. (1987). Student retention and the use of campus facilities by race. NASPA Journal, 24, 28-32.
Murtaugh, P. A., Burns, L. D., & Schuster, J. (1999). Predicting the retention of university students. Research in Higher Education, 40, 355-371.
Nagda, B. A., Gregerman, S. R., Jonides, J., von Hippel, W., & Lerner, J. S. (1998). Undergraduate student-faculty research partnerships affect student retention. The Review of Higher Education, 22, 55-72.
National Center for Education Statistics (2004). Student effort and educational progress. Retrieved May 25, 2005, from http://nces.ed.gov/programs/coe/2004/section 3/table.asp?tableID=61.
Peterson, S. L. (1993). Career decision-making self-efficacy and institutional integration of underprepared college students. Research in Higher Education, 34(6), 659-675.
Raudenbush, S. W., & Bryk, A. S. (2001). Hierarchical linear models: Applications and data analysis methods (2nd Ed.). Thousand Oaks, CA: Sage.
Sandier, M. E. (2000). Career decision-making self-efficacy, perceived stress, and an integrated model of student persistence: A structural model of finances, attitudes, behavior, and career development. Research in Higher Education, 41, 537-580.
Stage, F. K. (1989). Motivation, academic and social integration, and the early dropout. American Educational Research Journal, 26, 385-402.
Tinto, V. (1975). Dropout from higher education: A theoretical synthesis of recent research. Review of Educational Research, 45, 89-125.
Tinto, V. (1993). Leaving college: Rethinking the causes and cures of student attrition (2nd Ed.). Chicago, IL: University of Chicago Press.
Tinto, V., Russo, P., & Kadel, S. (1994). Constructing educational communities: Increasing retention in challenging circumstances. Community College Journal, 64, 26-30.
Wetzel, J. N., O'Toole, D., & Peterson, S. (1999). Factors affecting student retention probabilities: A case study. Journal of Economics and Finance, 23(1), 44-55.
Division of Educational Studies, University of Cincinnati
Policy and Evaluation Division, California Department of Education
Office of Institutional Research, University of Cincinnati
Department of Educational Research, Technology, and Leadership
University of Central Florida
Table 1 Descriptive Statistics by Program Type Total Variable Statistic (n = 1305) Age M 18.62 Min 16.68 Max 24.86 SD .56 Gender Female % .47 Male % .53 Ethnicity White % .83 Black % .11 Other % .06 HS GPA M 2.91 Min 1.20 Max 4.50 SD .64 College Selectivity: Not Selective % .32 Liberal Selective % .39 Moderate Selective % .11 High Selective % .18 Program Type Advising FYE Variable Statistic (n = 285) (n = 284) Age M 18.56 18.60 Min 16.68 17.92 Max 19.70 22.83 SD .42 .52 Gender Female % .53 .32 Male % .47 .68 Ethnicity White % .77 .90 Black % .16 .06 Other % .07 .04 HS GPA M 3.22 3.06 Min 1.62 1.87 Max 4.00 4.45 SD .47 .50 College Selectivity: Not Selective % .02 .03 Liberal Selective % .64 .38 Moderate Selective % .00 .51 High Selective % .35 .08 Program Type Social Academic General Integration Help Orientation Variable Statistic (n = 238) (n = 71) (n = 427) Age M 18.58 18.52 18.70 Min 17.72 17.44 17.68 Max 24.86 20.85 23.56 SD .65 .45 .63 Gender Female % .63 .24 .47 Male % .37 .76 .53 Ethnicity White % .88 .92 .79 Black % .05 .04 .16 Other % .07 .04 .05 HS GPA M 3.22 3.39 2.34 Min 1.97 2.26 1.20 Max 4.00 4.50 4.00 SD .46 .45 .54 College Selectivity: Not Selective % .01 .00 .94 Liberal Selective % .82 .18 .03 Moderate Selective % .00 .00 .00 High Selective % .17 .82 .03 Table 2 Fixed Effect Estimates from Multilevel Longitudinal Analysis for Retention Fixed Effect Coefficient SE For Intercept, [[pi].sub.0] For Intercept, [[beta].sub.00] Intercept, [[gamma].sub.000] -.135 .418 ACADEMIC HELP, [[gamma].sub.001] .997 .341 % of FEMALE, [[gamma].sub.002] -1.387 .422 For SELECTIVE, [[beta].sub.01] Intercept, [[gamma].sub.010] -.094 .103 ADVISING, [[gamma].sub.011] .537 .110 SOCIAL, [[gamma].sub.012] .487 .135 For FEMALE, [[beta].sub.02] Intercept, [[gamma].sub.020] .268 .135 For HIGH SCHOOL GPA, [[beta].sub.03] Intercept, [[gamma].sub.030] .412 .143 For ACADEMIC YEAR slope, [[pi].sub.1] For Intercept, [[beta].sub.10] Intercept, [[gamma].sub.100] -.869 .193 For HIGH SCHOOL GPA, [[beta].sub.11] Intercept, [[gamma].sub.110] .171 .065 Fixed Effect t df p For Intercept, [[pi].sub.0] For Intercept, [[beta].sub.00] Intercept, [[gamma].sub.000] -.323 17 .750 ACADEMIC HELP, [[gamma].sub.001] 2.924 17 .010 % of FEMALE, [[gamma].sub.002] -3.289 17 .005 For SELECTIVE, [[beta].sub.01] Intercept, [[gamma].sub.010] -.914 1301 .361 ADVISING, [[gamma].sub.011] 4.865 1301 .000 SOCIAL, [[gamma].sub.012] 3.601 1301 .001 For FEMALE, [[beta].sub.02] Intercept, [[gamma].sub.020] 1.988 1301 .046 For HIGH SCHOOL GPA, [[beta].sub.03] Intercept, [[gamma].sub.030] 2.873 1301 .005 For ACADEMIC YEAR slope, [[pi].sub.1] For Intercept, [[beta].sub.10] Intercept, [[gamma].sub.100] -4.498 19 .000 For HIGH SCHOOL GPA, [[beta].sub.11] Intercept, [[gamma].sub.110] 2.612 1303 .009 Note. SELECTIVE means that the FTFT student was in a selective college, and FEMALE and HIGH SCHOOL GPA are self-explanatory, % of FEMALE represents the percent of female participants in the intervention program, and ACADEMIC HELP, ADVISING, and SOCIAL are the types of intervention programs. Table 3 Fixed Effect Estimates from Multilevel Longitudinal Analysis for GPA Fixed Effect Coefficient SE For Intercept, [[pi].sub.0] For Intercept, [[beta].sub.00] Intercept, [[gamma].sub.000] .227 .087 ORIENTATION, [[gamma].sub.001] .374 .070 For SELECTIVE, [[beta].sub.01] Intercept, [[gamma].sub.010] .043 .029 SOCIAL, [[gamma].sub.011] .175 .026 For FEMALE, [[beta].sub.02] Intercept, [[gamma].sub.020] .090 .032 For WHITE, [[beta].sub.03] Intercept, [[gamma].sub.030] .306 .041 For HIGH SCHOOL GPA, [[beta].sub.04] Intercept, [[gamma].sub.040] .532 .033 For ACADEMIC YEAR slope, [[pi].sub.1] For Intercept, [[beta].sub.10] Intercept, [[gamma].sub.100] .119 .033 ORIENTATION, [[gamma].sub.101] -.061 .014 For WHITE, [[beta].sub.11] Intercept, [[gamma].sub.110] -.052 .015 For BLACK, [[beta].sub.12] Intercept, [[gamma].sub.120] -.078 .017 For HIGHSCHOOL GPA, [[beta].sub.13] Intercept, [[gamma].sub.130] -.024 .011 For RESIDENCE slope, [[pi].sub.2] For Intercept, [[beta].sub.20] Intercept, [[gamma].sub.200] .060 .013 For CREDIT HOURS slope, [[pi].sub.3] For Intercept, [[beta].sub.30] Intercept, [[gamma].sub.300] .002 .000 Fixed Effect t df p For Intercept, [[pi].sub.0] For Intercept, [[beta].sub.00] Intercept, [[gamma].sub.000] 2.609 18 .018 ORIENTATION, [[gamma].sub.001] 5.330 18 .000 For SELECTIVE, [[beta].sub.01] Intercept, [[gamma].sub.010] 1.497 1300 .134 SOCIAL, [[gamma].sub.011] 6.690 1300 .000 For FEMALE, [[beta].sub.02] Intercept, [[gamma].sub.020] 2.789 1300 .006 For WHITE, [[beta].sub.03] Intercept, [[gamma].sub.030] 7.498 1300 .000 For HIGH SCHOOL GPA, [[beta].sub.04] Intercept, [[gamma].sub.040] 16.005 1300 .000 For ACADEMIC YEAR slope, [[pi].sub.1] For Intercept, [[beta].sub.10] Intercept, [[gamma].sub.100] 3.610 18 .002 ORIENTATION, [[gamma].sub.101] -4.357 18 .000 For WHITE, [[beta].sub.11] Intercept, [[gamma].sub.110] -3.474 1301 .001 For BLACK, [[beta].sub.12] Intercept, [[gamma].sub.120] -4.522 1301 .000 For HIGHSCHOOL GPA, [[beta].sub.13] Intercept, [[gamma].sub.130] -2.098 1301 .036 For RESIDENCE slope, [[pi].sub.2] For Intercept, [[beta].sub.20] Intercept, [[gamma].sub.200] 4.556 3014 .000 For CREDIT HOURS slope, [[pi].sub.3] For Intercept, [[beta].sub.30] Intercept, [[gamma].sub.300] 14.882 3014 .000 Note. RESIDENCE means whether the student lived in a dormitory, and CREDIT HOURS is the cumulative credit hours. SELECTIVE means that the FTFT student was in a selective college, and other predictors are self-explanatory. ORIENTATION and SOCIAL are the types of intervention programs.
|Printer friendly Cite/link Email Feedback|
|Author:||Pan, Wei; Guo, Shuqin; Alikonis, Caroline; Bai, Haiyan|
|Publication:||College Student Journal|
|Date:||Mar 1, 2008|
|Previous Article:||Student characteristics and expectations of university classes: a free elicitation approach.|
|Next Article:||Some thoughts on the issue of making the liberal science courses more appealing to academics.|