Two tests of the effectiveness of bilingual education in preschool.
Abstract. Three meta-analytic studies have shown that bilingual education is an effective method for teaching students who are English language learners. However, there is limited evidence of the effectiveness of bilingual education in preschool. This study used multiple years of data from the Manchester (New Hampshire) Even Start program and relevant comparison groups to conduct two separate tests of the effectiveness of bilingual education in preschool. The results support the effectiveness of bilingual education in preschool, but only at marginally significant levels of statistical inference. A fixed effects model of literacy outcomes comparing a cohort of students who had received bilingual education in preschool but English-only education in kindergarten with a cohort of students who received English-only education in both preschool and kindergarten showed evidence of an achievement gap (favoring the students receiving bilingual education) that emerged at the end of preschool but had closed by the end of kindergarten (10 values ranged from .028 to .068). Also, an analysis of two years of preschool data from an Even Start program that utilized bilingual education in the first year, but not the second, showed that Even Start students who did not receive bilingual education performed worse on posttest literacy assessments when controlling for pretest scores (p values ranged from .056 to .060). However, when estimated with a control group consisting of only English language learners, this negative effect was no longer significant. Although the study provides some evidence of the effectiveness of bilingual education in preschool, further research is needed to validate these findings.
Mounting evidence continues to demonstrate the effectiveness of bilingual education in improving the academic performance of English language learners. Three major meta-analytic studies have shown statistically significant positive effects of bilingual education programs relative to English-only education for English language learners. The earliest was an analysis of 23 kindergarten, primary, and secondary grade bilingual education interventions in the United States, conducted by Willig (1985). The study found an overall "small to moderate" statistically significant positive effect for bilingual education programs. For tests administered in English, Willig found significant effects favoring bilingual education for reading in English, language in English, math in English, and total achievement in English. The study also identified that effect sizes tended to be larger when, among other things, the study used random assignment, when both the experimental and control groups were Spanish-dominant, when well-performing students were not removed from the experimental groups, and when comparison students had not previously been in bilingual classrooms.
The second major meta-analysis was conducted by Greene (1998), who used a subset of 11 studies from the Rossell and Baker (1996) review, which had argued that bilingual education programs tended to be less effective than English-only education. In contrast to the Rossell and Baker findings, Greene found a small to moderate positive effect of bilingual education programs on educational achievement, also showing that studies using random assignment resulted in larger effect sizes of bilingual education. The most recent meta-analysis was conducted by Rolstad, Mahoney, and Glass (2005) and consisted of 17 studies of K-12 students published after Willig's (1985) study. This study had results similar to those found in the Greene and Willig examinations; namely, that bilingual education is "consistently superior to all-English approaches" (p. 572). Also, consistent with findings from the Willig study, Rolstad et al. found that effect sizes were higher when experimental students were compared to control students who were English language learners, as opposed to native English speakers.
While these studies convincingly demonstrate the effectiveness of bilingual education in a kindergarten though grade 12 student population, much less evidence exists regarding the effectiveness of bilingual education in a preschool setting. Notably absent from the aforementioned meta-analyses are studies that assessed the impact of bilingual education among preschool students. Most of the studies included in the three meta-analyses examined the effects of bilingual education among elementary school students, several of the studies (particularly from the Willig analysis) included kindergarten students, and relatively few of the studies included students in grades 9 or later. The Willig (1985) and Rolstad et al. (2005) studies focused explicitly on K-12 bilingual programs and the Greene (1998) study did not include any studies with preschool students.
Of the studies examining the effectiveness of bilingual education in preschool, a study by Rodriguez, Diaz, Duran, and
Espinosa (1995) found that Latino students attending bilingual preschool showed greater language improvement than a control group of Latino children who did not attend preschool. In a replication of the Rodriguez et al. study, Winsler, Diaz, Espinosa, and Rodriguez (1999) showed a similar positive effect of bilingual preschool in the improvement of language proficiency among Latino preschool students, relative to a control group that did not attend any preschool. A related analysis used follow-up assessments from the cohort of students from the 1995 Rodriguez study in the year following the initial study and found parallel improvement among the program and control groups, demonstrating that the students who had attended bilingual preschool had maintained their superior English language proficiency one year after the initial intervention (Winsler et al., 1999).
While showing significant program effects, these studies and others that have assessed the effectiveness of bilingual education in preschool (e.g., McConnell, 1980) have used untreated control groups, instead of control groups consisting of students who attended English-only preschool. Consequently, it is uncertain from these studies whether the students in bilingual preschool would have shown similar language improvement relative to the controls, had they attended an English-only preschool.
The Current Study
Building on a previous study, the current study uses two additional data sources to evaluate the association between preschool bilingual education and literacy outcomes. In the initial study (the "first wave" analysis), it was found that family participation in the Manchester, New Hampshire, Even Start program was associated with preschoolers' posttest scores on the PALS-Pre-K assessment, which were significantly higher than those of a comparison group when controlling for pretest scores (Ryan, 2005). (Even Start is a well-known, nationally implemented family literacy program. See www.evenstart.org for details.) In this study, the program group consisted of 4-year-old Latino students from low-income families whose families were enrolled in the Even Start program, while the comparison group consisted of ethnically and linguistically diverse 4-year-old preschool students from low-income families. All of the students in the study received preschool education at the same site.
From the first wave analysis, the elements of the Even Start program that were identified as being the most likely source of the improvement were the bilingual education approach to preschool education and regular home visits from culturally and linguistically concordant outreach workers. The instruction in the Even Start class used a "bridging" approach, whereby a bilingual co-teacher would integrate the use of Spanish to facilitate student understanding of the otherwise English-only instruction (see Ryan, 2005). As the students' language skills improved, the use of Spanish was gradually reduced over the course of the school year. This mode of bilingual education corresponds approximately to the sheltered English immersion (Freeman & Freeman, 1988) or early exit (Rennie, 1993) models of bilingual education. Students in the comparison group attended English-only preschool in the same Title I preschool as the Even Start students. (Title I is a federal education program that provides financial resources for a variety of interventions, including preschool aimed at disadvantaged students. See www.ed.gov/ policy/elsec/leg/esea02/pgl.html for details.) A Title I preschool is similar to Even Start in terms of the qualifications of teachers, recommended curricula, and time in the classroom, as well as the income level of the students' families. Other differences in the intervention received by the Even Start students, versus that which was received by the comparison group, were that families in the Even Start program were required to receive home visits, participate in adult education, and participate in parent and child interactive literacy activities. Details about the community context and classroom setting for these interventions are described elsewhere (Ryan, 2005).
The current study builds on the initial study by measuring whether the improvement in preschool literacy outcomes for Latino students enrolled in the Manchester Even Start program, relative to a control group, was maintained through the following year of English-only kindergarten. This analysis will be called the "continuation analysis." It will also assess whether a replication of the initial Even Start intervention in year 2, which did not include a preschool bilingual education component, resulted in improvement among enrolled Even Start students that was comparable to that which was observed in the first wave analysis. This analysis will be called the "second wave analysis."
These questions can be assessed because the Manchester Even Start intervention changed for children and their families in two important ways between the 2003-04 and 2004-05 program years. One change was that Even Start students from the first wave analysis who had previously received bilingual education together in a preschool classroom were enrolled in standard English-only kindergarten classrooms in public elementary schools across Manchester. While the classroom educational experience of the Even Start students changed, the home visiting and parent and child interactive literacy activities remained similar. Thus, if Even Start students from the first wave analysis maintained, or increased, their literacy achievement gap relative to the comparison group during their kindergarten year, then it is possible that the first wave achievement gap was not a result of bilingual education (and instead could be attributed to other Even Start programmatic elements). However, if the first wave achievement gap was reduced or eliminated over the kindergarten year (when the Even Start students did not receive bilingual education), the effectiveness of preschool bilingual education would be supported.
The second important change in the 2004-05 Even Start program was that bilingual education was not implemented among the incoming 4-year-old preschool students. Also, while these incoming students and their families were required to be English language learners and received a family literacy intervention similar to that received by the first wave Even Start preschool students, these students were not required to be Spanish-speaking. Another change to the intervention was that the Even Start students attended preschool at a different site than they had in the 2003-04 program year (albeit with the same main classroom teacher from the previous year), while the control students remained at the same site. Differences in the program components and participant characteristics over the 2003-04 and 2004-05 program years are summarized in Table 1.
Because preschool bilingual education was not implemented in the 2004-05 program year while the other program elements remained similar, if Even Start preschool students in the 2004-05 program year did not show as much improvement as Even Start students from the 2003-04 program year relative to the comparison groups from each year, then the effectiveness of preschool bilingual education would be supported.
This article will use the changes in the Even Start program in the 2003-04 and 2004-05 program years to conduct two separate tests of the effectiveness of bilingual education in preschool, using follow-up kindergarten data on the original first wave cohort as well as additional data from the new, second wave cohort of preschoolers.
The study utilized a retrospective repeated measures cohort design, where literacy performance was compared among students who had received different types of early childhood instruction, but had not been assigned to their cohort. In the continuation analysis, three observations (preschool pretest, preschool posttest, and kindergarten posttest) were compared for two cohorts: one that received the Even Start intervention in preschool and kindergarten, and one that received standard preschool and kindergarten.
The second wave analysis builds on the design of the first wave data by incorporating additional pretest and posttest data from two new cohorts: 2004-05 four-year-old Even Start students and 2004-05 four-year-old comparison group students. In the second wave analysis, two observations (preschool pretest and pretest posttest) are available for both Even Start preschool students and Title I preschool students in both the 2003-04 and 2004-05 program years.
The measures used to assess preschool and kindergarten literacy performance were the PALS-Pre-K (Invernizzi, Sullivan, & Meier, 2001) and PALS-K (Invernizzi, Swank, & Juel, 2001) assessments, respectively. The PALS-Pre-K is a measure of emergent literacy. Although subsequently revised, the version used in the current study consisted of six subscales (rhyme awareness, uppercase alphabet knowledge, beginning sound awareness, verbal memory, print knowledge, and name writing) that are added together to obtain a total score. The instrument has been shown to have high reliability and validity (Invernizzi, Sullivan, Meier, & Swank, 2004; Ryan, 2005). The PALS-K is a similar measure of phonological awareness and literacy skills, with established reliability and validity, aimed at kindergarten students (Invernizzi, Juel, Swank, & Meier, 2004). Tests were administered by classroom teachers trained in the testing procedures.
In the 2003-04 program year, 52 students (including 14 Even Start students) had matched pretests and posttests. Two students in the Even Start classroom were excluded from the current analyses because their parents did not fully participate in the intervention, leaving a total of 50 students with matched pretests and posttests. Of these 50 students, 40 students completed the PALS-K assessment at the end of their kindergarten year (pretests were not administered).
For the second wave analysis, 139 students took the PALS-Pre-K pretest and 129 students (including 15 Even Start students) had matched pretests and posttests. Sixteen students were identified as having special needs and were excluded from the second wave study (as similarly identified students were excluded from the first wave study), resulting in a total of 163 students (50 from the first wave + 113 from the second wave) pooled together in the second wave analysis.
In addition to these assessments, program records were obtained from both years of the Even Start program, which included information on the number of home visits, adult education attendance, number of parent and child interactive literacy events attended, and school attendance. Programmatic data available for students in the continuation analysis and second wave analysis are shown in Table 2. The first four columns contain data that is relevant to the continuation analysis, while the first, second, fifth, and sixth columns contain data that is relevant to the second wave analysis.
Table 2 shows that the number of home visits for Even Start parents and their children in the continuation analysis was similar across the 2003-04 and 2004-05 program years, while the number of parent and child interactive literacy events (PCIL) and the number of English for speakers of other language (ESOL) classes attended by families differed. For Even Start students and their families in the second wave analysis, preschool attendance and the number of home visits received were similar across the 2003-04 and 2004-05 program years, but the number of PCIL events attended differed, with attendance being greater in 2004-05. Also worth noting in Table 2 is that programmatic data is missing for four Even Start students and their families in the 2004-05 kindergarten year. This lack of data resulted from the fact that two families participated in Even Start for only half of the program year, while two other families did not participate at all during the program year.
For the continuation study, fixed effects regression was used to model literacy growth for the unbalanced panel of students across the three observation periods. The fixed effects model is useful in this context, in that it controls for unobserved and time invariant factors at the student level (e.g., race, gender, parent education, parent motivation), which may relate to test performance by allowing each student to have his or her own intercept in the analysis. A fixed effects model is also preferable to a random effects model in this analysis, because the fixed effects model coefficients remain unbiased in the presence of correlation between the individual effect ([a.sub.i] in Equation 1) and other variables in the model, while random effects coefficients are biased in the presence of such correlation (Greene, 2003). A Hausman test confirmed that the model coefficients estimated by fixed effects and random effects were significantly different (p < .10), providing evidence of a bias in the random effects coefficients.
The fixed-effects model specified in the analysis is shown in Equation 1.
Equation 1. Specification of fixed effects model for continuation analysis
[Y.sub.it] = [[beta].sub.1] Year1 [posttest.sub.it] + [[beta].sub.2] Year2 [posttest.sub.it] + [[beta].sub.3] [Intervention.sub.i.sup.*] [Yearlposttest.sub.it] + [beta].sub.4] [Intervention.sub.i] * Year2 [posttest.sub.it] + [a.sub.i] + [e.sub.it]
[Y.sub.it] = literacy score for student i at time t Year1 [posttest.sub.it]-- dummy variable indicating that a preschool posttest was taken by student i at time t
Year2 posttestit = dummy variable indicating that a kindergarten posttest was taken by student i at time t
[Intervention.sub.i.sup.*] Year1 [postest.sub.it] = interaction term indicating the joint participation in Even Start for student i and that a preschool posttest was taken by student i at time t
[Intervention.sub.i.sup.*] Year2 [posttest.sub.it] = interaction term indicating the joint participation in Even Start for student i and that a kindergarten posttest was taken by student i at time t
[a.sub.i] = fixed effect for student i
[e.sub.it] = error term for student i at time t
The coefficients of interest in this model are [[beta].sub.3] and [[beta].sub.4], the two interaction terms between the intervention and the posttests. The coefficient [[beta].sub.3] indicates whether students in the Even Start program had higher literacy scores than comparison students in year 1 when controlling for the individual fixed effect. Similarly, the coefficient [[beta].sub.4] indicates whether students in the Even Start program had higher literacy scores than comparison students in year 2 when controlling for the individual fixed effect. If [[beta].sub.3] is significantly larger than [[beta].sub.4], then the effectiveness of bilingual education is supported, because the use of English-only instruction for Even Start students in their kindergarten year was the primary change across program years. A test of the equivalence of [[beta].sub.3] and [[beta].sub.4] was performed by using a Wald test.
Two alternative approaches were taken to estimating the standard errors of the coefficients in the specified model. The first approach used robust, or White, standard errors, to estimate standard errors that are robust to heteroskedastic errors (Greene, 2003). The second approach estimated bootstrap standard errors, from 1,000 bootstrap replications, in order to calculate standard errors based on the empirical properties of the data and not their theoretical distributions (Greene, 2003). Bootstrapping is particularly relevant in relatively small samples, when parameter inference is sensitive to the normality of errors.
For the second wave study, data from the 2003-04 and 2004-05 program years were pooled. A regression model of the following form was specified:
Equation 2. Specification of regression model for second wave analysis
[Y.sub.i] = [[delta].sub.0] + [[delta].sub.1] + [[delta].sub.2] wave [2.sub.i] + [[delta].sub.3] [Intervention.sub.i] + [[delta].sub.4] [Intervention.sub.i] * wave [2.sub.i] + [e.sub.i]
[Y.sub.i] = posttest score for student i
[pretest.sub.i] = pretest score for student i
wave [2.sub.i] = dummy variable indicating that student i attended preschool in the 2004-05 program year
[Intervention.sub.i] = dummy variable indicating that student i participated in Even Start in either program year
[Intervention.sub.i.sup.*] [wave2.sub.i] = interaction term indicating the participation in Even Start for student i in the 2004-05 program year [e.sub.i] = error term for student i
The coefficients of interest in the model are [[delta].sub.3] and [[delta].sub.4]. If [[delta].sub.3] is positive and significant, then Even Start students in wave 1 had higher posttest scores than comparison students when controlling for pretest scores. If [[delta].sub.4] is negative and significant, then Even Start students in the second wave had lower posttest scores than Even Start students in the first wave when controlling for pretest scores. A negative coefficient on [[beta].sub.4] would thus support the effectiveness of bilingual education in preschool.
To standardize the test scores across the different instruments and to facilitate parameter interpretation, raw test scores were converted to z-scores for each test administration for use in the regression analyses.
All analyses were performed using Stata version 9.1.
Table 3 shows the results of the fixed effects model for the continuation analysis. The coefficient on the Intervention * Post 1 interaction term ([[beta].sub.3]) indicates that students in the Even Start program are predicted to score .83 standard deviations higher on the PALS-Pre-K, relative to students in the comparison group at the end of their preschool year. This coefficient is significant at p = .001 when robust standard errors are used and at p = .007 when bootstrap standard errors are used. The coefficient on the Intervention * Post 2 interaction term ([[beta].sub.4]) indicates that students in the Even Start program are predicted to score .07 standard deviations higher on the PALS-K than students relative to the comparison group at the end of their kindergarten year. This coefficient is not close to significance, using either standard error estimator.
A Wald test was performed to assess whether the Intervention * Post 1 interaction and the Intervention * Post 2 interaction were equal. Equality of the two interaction coefficients was rejected at p = .028 using the robust standard errors and at p = .068 using the bootstrap standard errors.
Table 4 shows the results of the regression model specified for the second wave analysis. The coefficient for the Intervention variable ([[delta].sub.3]) is .66 and the coefficient for the Intervention * wave 2 interaction ([[delta].sub.4]) is -.54. The Intervention coefficient is significant at p = .006 with robust standard errors and at p = .007 with bootstrap standard errors. The Intervention * wave 2 interaction is significant at p = .056 and p = .060 when using the robust and bootstrap standard errors, respectively.
Table 5 shows the results of the same regression run among a sub-sample of-preschool English language learners across the two program years. Here, the coefficient for the Intervention variable is slightly smaller and the Intervention and wave 2 interaction coefficient is slightly greater than the coefficient that was estimated using the entire sample. The coefficient of the Intervention * wave 2 interaction is not significant when either robust (p = .196) or bootstrap (p = .189) standard errors are estimated.
Using results from two separate sets of analyses, this study found evidence supporting the effectiveness of bilingual education in a low-income preschool setting, but only at marginally significant levels of statistical inference. The analysis of literacy outcome data from a cohort of students who had received bilingual education in preschool, but English-only education in kindergarten, compared with a cohort of students who received English-only education in both preschool and kindergarten, showed evidence of an achievement gap (favoring the students receiving bilingual education) that emerged at the end of preschool but had closed by the end of kindergarten. A test for the equality of the achievement gaps at the end of preschool and the end of kindergarten was rejected at p = .028 and p = .068, depending on the specification of the standard errors in the fixed effects model.
The analysis of two years of preschool data from Even Start programs that utilized bilingual education in the first year, but not the second, showed that Even Start students who did not receive bilingual education performed worse on posttest literacy assessments when controlling for pretest scores (p = .056 when robust standard errors were estimated and p = .060 when bootstrap standard errors were estimated). However, when estimated with a control group consisting of only English language learners, this negative effect was no longer significant (p = .196 when robust standard errors were estimated and p = .189 when bootstrap standard errors were estimated).
Thus, results from the analysis provide some support for the effectiveness of bilingual education in preschool, but only at marginally significant levels of statistical inference. The continuation analysis found evidence of a bilingual education effect at p < .05, but this result was sensitive to the method of standard error estimation. Similarly, the second wave analysis found evidence of a bilingual education effect at p < .10 among a complete sample of students, but not among a sub-sample of English language learners.
The study has several relevant limitations. As in all retrospective cohort designs, the selection threat is possible due to differences in the groups who self-select to participate in an intervention versus those who do not. To address this issue, individual fixed effects (accounting for time-invariant unobserved factors at the student level) were employed in the continuation analysis and pretest scores, indicative of literacy ability, were controlled for in the second wave analysis. In addition, the results of the continuation study, which showed a year 1 cross-over effect for Even Start students but a subsequent year 2 regression relative to the comparison students, weakens the selection threat because, if present, the threat would most likely be consistent across the time periods, which was not evidenced in the data.
Another threat to the study's internal validity is the testing threat, which manifested itself in the first wave study when Even Start students were tested on the PALS-Pre-K at midyear (in addition to pre and post), while the comparison group students did not take the midyear assessment. While there is no way to rule out this threat, common sense indicates that this alone is unlikely to result in a substantial change in posttest scores, particularly given that the midyear and posttest assessments were administered several months apart.
The other relevant threat to internal validity was maturation, which could have occurred in the continuation study. This threat would be relevant if a natural drop-off in performance could be expected among students in bilingual education interventions after an initial period of success. If true, Even Start students in their kindergarten year of the continuation study would be expected to lose their literacy performance advantage relative to the comparison group, even if bilingual education had been maintained. However, evidence from other studies suggests that a second year drop-off is not the natural consequence of bilingual education interventions. The Winsler et al. study showed that students who received bilingual preschool maintained their proportional advantage to control students a year after the intervention. Also, a longitudinal study by Ramirez, Yuen, Ramey, Pasta, and Billings (1991) suggests that English language learners in English immersion classes may fall further behind their English-speaking peers than students in bilingual education. Consequently, it appears unlikely that the relative performance of Even Start and Title I students in the continuation study was the result of a natural maturation process.
A further limitation was related to the fact that four of the families of students from the first wave analysis did not fully participate in the Even Start intervention during their child's kindergarten year (with two families not participating at all and two others participating partially). If the elements of the Even Start intervention other than bilingual education (e.g., home visiting; parent and child interactive literacy events) were relevant in impacting student literacy outcomes in kindergarten, then the lack of participation by these families may have reduced student literacy achievement, in addition to any reduction that may have occurred as a result of not receiving bilingual education.
A final issue in the study relates to the statistical inferences made. Given the small sample size, particularly among the students receiving bilingual education, the analyses had low power to reject the null hypotheses posed in the analyses. Thus, the study suffers from a high probability of type II error: failing to reject the null hypotheses when, in fact, they should be rejected.
This study contributes to the existing literature in bilingual education by providing two relevant quantitative tests of the effectiveness of bilingual education in preschool. The study design, employing comparison groups who received English-only education, is better suited to assess the effect of bilingual education than previously published studies using comparison groups of students who did not attend preschool. In addition, the use of the fixed effects estimator strengthens the causal inference by accounting for unobserved time-invariant factors at the student level, which may confound inference if left unaddressed. Bootstrap standard errors also may provide a more accurate reflection of parameter variance by avoiding distributional assumptions about the error term. Although this study suggests that bilingual education in preschool may improve preschool literacy outcomes, further research is necessary to validate this finding.
Freeman, D., & Freeman, Y. (1988). Sheltered English instruction. ERIC Digest. Washington, DC: ERIC Clearinghouse on Languages and Linguistics. (ERIC Document Reproduction Service No. ED301070)
Greene, J. P. (1998). A meta-analysis of the effectiveness of bilingual education. Claremont, CA: Thomas Rivera Policy Institute.
Greene, W. H. (2003). Econometric analysis (5th ed.). East Rutherford, NJ: Prentice Hall.
Invernizzi, M., Juel, C., Swank, L., & Meier, J. (2004). PALS-K 2004-2005 Technical Reference. Charlottesville, VA: University of Virginia Press.
Invernizzi, M., Sullivan, A., & Meier, J. (2001). Phonological Awareness Literacy Screening: PALS-Pre-K. Charlottesville, VA: University of Virginia Press.
Invernizzi, M., Sullivan, A., Meier, J., & Swank, L. (2004). PALS-Pre-K teacher's manual. Charlottesville, VA: University of Virginia Press.
Invernizzi, M., Swank, L., & Juel, C. (2001). Phonological Awareness Literacy Screening: Kindergarten (2nd ed.). Charlottesville, VA: University of Virginia Press.
McConnell, B. (1980). Individualized bilingual instruction. Final evaluation: 1978-1979 program year. Pullman, WA: Individualized Bilingual Instruction Evaluation Office. (ERIC Document Reproduction Service No. ED211305)
Ramirez, J. D., Yuen, S. D., Ramey, D. R., Pasta, D.J., & Billings, D. (1991). Longitudinal study of structured English immersion strategy, early-exit and late-exit transitional bilingual education programs for language-minority children. Final Report. San Mateo, CA: Aguirre International. (ERIC Document Reproduction Service No. ED330216)
Rennie, J. (1993). ESL and Bilingual Program Models. ERIC Digest. Washington, DC: ERIC Clearinghouse on Languages and Linguistics. (ERIC Document Reproduction Service No. ED362072)
Rodriguez J. L., Diaz, R. M., Duran, D., & Espinosa, L. (1995). The impact of bilingual education on the language development of Spanish-speaking children. Early Childhood Research Quarterly, 10(4), 475-490.
Rolstad, K., Mahoney, K., & Glass, G.V. (2005). The big picture: A meta-analysis of program effectiveness research on English language learners. Educational Policy, 19(4), 572594.
Rossell, C. H., & Baker, K. (1996). The educational effectiveness of bilingual education. Research in the Teaching of English, 30(1), 7-74.
Ryan, A. M. (2005). The effectiveness of the Manchester Even Start program in improving literacy outcomes for preschool Latino students. Journal of Research in Childhood Education, 20(1), 15-26.
Willig, A. C. (1985). A meta-analysis of selected studies on the effectiveness of bilingual education. Review of Educational Research, 55(3), 269-318.
Winsler A., Diaz, R. M., Espinosa, L., & Rodriguez, J. L. (1999). When learning a second language does not mean losing the first: Bilingual language development in low-income, Spanish-speaking children. Child Development, 70(2), 349-362.
Andrew M. Ryan
The Heller School of Social Policy Brandeis University
Author Note. I thank Sheila Brisson, Kathy Anibal, Kathy Erskin, and Joanne Mulligan at the Manchester School District for assistance in the data collection for this study. I also thank Christina Rossell for providing information about a study in her literature review.
Table 1 Description of Preschool and Kindergarten Program Components and Participant Characteristics in 2003-04 and 2004-05 Program Years 2003-04 2004-05 Preschool * Bilingual preschool * English-only preschool intervention * Home visiting * Home visiting group * Parent and child * Parent and child interactive literacy interactive literacy * Adult education * Adult education * Students and families * Students and families are are low-income and low-in-come and English Spanish-speaking language learners with multiple native languages Preschool * English-only preschool * English-only preschool comparison * Students and families * Students and families are group are low-income and have low-income and have multiple native multiple native languages; some are languages; some are English language English language learners learners Kindergarten * English-only kindergarten intervention * Home visiting group * Parent and child interactive literacy * Adult education * Students and families are low-in-come and are Spanish-speaking Kindergarten * English-only kindergarten comparison * Students and families are group low-income and have multiple native languages; some are English language learners Table 2 Characteristics of Students Included in Second Wave and Continuation Analysis 2003-04 Preschool Preschool Kindergarten intervention comparison intervention (n=12) (n=38) (n=10) (a) Mean pre- 26.3 (11.3) 42.7 (22.9) NA test score (s.d.) Mean post- 74.3 (19.3) 72.7 (22.9) 77.5 (13.9) test score (s.d.) Mean days 112.8 (9.9) 122.1 (b) NA attendance (10.4) (s.d.) Number of home visits (n) 0-10 25% (3) NA 33.3% (2) 11-15 41.7% (5) 50% (3) 16-highest 33.3% (4) 16.7% (l) Number of PCIL events attended (n) 0-4 41.7% (5) 0 5-10 58.3% (7) NA 50% (3) 11-15 0 33.3% (2) 16-highest 0 16.7% (l) Number of parental ESOL classes attended (n) 0-4 16.7% (2) 16.7% (l) 5-10 25% (3) NA 0 11-20 25% (3) 50% (3) 21-highest 33.3% (4) 33.3% (2) 2004-05 Kindergarten Preschool Kindergarten comparison intervention comparison (n=30) (n=15) (n=97) Mean pre- NA 46.1 (17.0) 48.4 (21.0) test score (s.d.) Mean post- 90.1 (20.2) 76.7 (11.9) 76.0 (23.1) test score (s.d.) Mean days NA 109.2 (15.2) 105.5 (20.9) attendance (s.d.) Number of home visits (n) 0-10 20% (3) NA 11-15 NA 46.7% (7) 16-highest 33.3% (5) Number of PCIL events attended (n) 0-4 33.3% (5) 5-10 NA 6.7% (1) NA 11-15 26.7% (4) 16-highest 33.3% (5) Number of parental ESOL classes attended (n) 0-4 5-10 NA NA (c) NA 11-20 21-highest (a) Four students whose families had participated in Even Start during the child's preschool year did not participate during the child's kindergarten year (b) Seven students had missing data (c) Data was not available for parental ESOL attendance for this cohort Table 3 Fixed Effects Regression Results Using Z-scores of Preschool and Kindergarten Literacy Achievement From Continuation Analysis Coefficient Robust standard error Year 1 Posttest ([[beta].sub.1]) -.20 .15 Year 2 Posttest ([[beta].sub.2]) .08 .18 Intervention * .83 .25 Year 1 Posttest ([[beta].sub.3]) Intervention * .07 .30 Year 2 Posttest ([[beta].sub.4]) P value Bootstrap P value standard error Year 1 Posttest ([[beta].sub.1]) .183 .18 .259 Year 2 Posttest ([[beta].sub.2]) .657 .20 .697 Intervention * .001 .31 .007 Year 1 Posttest ([[beta].sub.3]) Intervention * .821 .36 .848 Year 2 Posttest ([[beta].sub.4]) Note. p = .028 for Wald test that Even Start * Post 1 = Even Start * Post 2 using standard errors robust to heterokedasticity; p = .068 for Wald test that Even Start * Post 1 = Even Start * Post 2 using bootstrap standard errors Note. [R.sup.2] within students = .10, [R.sup.2] between students = .06 Number of observations = 140. Number of groups = 50 Table 4 Linear Regression Results Using Z-scores of Preschool Literacy Achievement From Second Wave Analysis (N=163) Coefficient Robust standard error Pretest ([[delta].sub.1]) .76 .06 Wave 2 ([[delta].sub.2]) -.05 .14 Intervention ([[delta].sub.3]) .66 .24 Intervention * Wave2 ([[delta].sub.4]) -.54 .28 Constant ([[delta].sub.0]) -.03 .13 P value Bootstrap P value standard error Pretest ([[delta].sub.1]) <.001 .06 <.001 Wave 2 ([[delta].sub.2]) .710 .15 .718 Intervention ([[delta].sub.3]) .006 .24 .007 Intervention * Wave2 ([[delta].sub.4]) .056 .29 .060 Constant ([[delta].sub.0]) .797 .13 .803 Note. [R.sup.2] = .5393 Table 5 Linear Regression Results Using Z-scores of Preschool Literacy Achievement From Second Wave Analysis for Only English Language Learners (N=63) Coefficient Robust standard error Pretest .70 .12 Wave 2 -.03 .24 Intervention .60 .28 Intervention * Wave 2 -.50 .38 Constant -.03 .14 P value Bootstrap P value standard error Pretest <.001 .12 <.001 Wave 2 .889 .24 .886 Intervention .037 .28 .032 Intervention * Wave 2 .196 .38 .189 Constant .822 .14 .817 Note. R2 =.4759