Approaches to Incorporating Late Pretests in Experiments: Evaluation of Two Early Mathematics and Self-Regulation Interventions.
Author(s): Unlu, Fatih; Layzer, Carolyn; Clements, Douglas; Sarama, Julie; Cook, David
Many educational Randomized Controlled Trials (RCTs) collect baseline versions of outcome measures (pretests) to be used in the estimation of impacts at posttest. Although pretest measures are not necessary for unbiased impact estimates in well executed experimental studies, using them increases the precision of impact estimates and reduces sample size requirements to detect desired effect sizes. This paper provides a theoretical overview of the late pretest issue and empirically assesses the bias-precision tradeoff in the experimental evaluation of the efficacy of adding a curriculum designed to support the development of children's self-regulation skills (Scaffolded Self-Regulation, SSR) to an established early mathematics curriculum (Building Blocks, BB) to form a synthesized curriculum, BBSSR. The RCT described in this paper is conducted in three large districts in Southern California. The analytic sample includes 807 students who participated in both baseline and posttest data collection. These students were in 84 4-year-old classrooms across the three districts. The authors analyzed a three-armed cluster RCT in which classrooms in early childhood centers or schools were randomized to the two intervention conditions and a business-as-usual control condition. Randomization was conducted separately for schools with one or two participating classrooms (groups A and B). Schools/centers in group A were placed into five randomization blocks such that each block consisted of all half-day or full-day PreK classrooms in one of the three study districts. Within each block, schools/centers were sorted with respect to prior math achievement, % reduced price lunch eligible, and %ELL and randomly assigned to the three conditions three at a time, starting with a randomly chosen point in the sorted list. This process was used to ensure balanced experimental groups. In each group B center/school, the two study classrooms were randomly assigned to two conditions that were randomly determined. Analyses conducted thus far show that controlling for contaminated pretest measures in impact regressions can lead to substantially biased impact estimates while their effect on the precision of impact estimates is less profound, causing one to question the overall merit of using them. Figures and exhibits are appended.
ERIC Descriptors: Mathematics Instruction; Intervention; Pretests Posttests; Curriculum Design; Control Groups; Experimental Groups; Instructional Effectiveness; Metacognition; Skill Development; Mathematics Curriculum; School Districts; Mathematics Achievement; Early Childhood Education; Scaffolding (Teaching Technique); Child Care Centers; Statistical Analysis; Task Analysis; Longitudinal Studies; Surveys; Children; Measures (Individuals); Child Development; Expressive Language; Language Tests; Preschool Children
Publisher: Society for Research on Educational Effectiveness. 2040 Sheridan Road, Evanston, IL 60208. Tel: 202-495-0920; Fax: 202-640-4401; e-mail: firstname.lastname@example.org; Web site: http://www.sree.org
Source: Society for Research on Educational Effectiveness
ERIC Number: ED563036
Record Type: Non-Journal
Publication Type: Reports - Research
Education Level: Early Childhood Education
|Printer friendly Cite/link Email Feedback|
|Author:||Unlu, Fatih; Layzer, Carolyn; Clements, Douglas; Sarama, Julie; Cook, David|
|Date:||Jan 1, 2013|
|Previous Article:||Effect of Observation Mode on Measures of Teaching.|
|Next Article:||Making Sense of the "Zero Effect" of Comprehensive Teacher Induction Programs: A Mediational Analysis.|