Printer Friendly

Does that program really work? Six questions to ask about research.

At MIND Research Institute, we have focused intently on studying the real-world, at-scale effectiveness of ST Math, our visual math software. In the process, we've gained a perspective unique among publishers on what constitutes valid effectiveness research. That's why we offer these guidelines on how to assess research about K-12 curricular programs.

1. What was the size of the study? Some of the results need to be at substantial scale, meaning at least five school-wide (not classroom) implementations. Super-motivated teachers can and will get outperforming results using ANY program. With large-scale studies (e.g. MIND has several from 25-50 schools) there are a substantial number of teachers involved, and it's unlikely they are all champions.

2. Are the results repeatable over years? Only having one year of results is not compelling. It's very important to see whether improvement is maintained or even grows in Year Two, and to see that two different sets of students achieved similar results.

3. What were the results compared to? Just having outcomes improve with program use means nothing; the improvement must be greater than a similar situation without the program. Also, the added improvement from the program must withstand a test of statistical significance.

4. Are the results repeatable across states? Look for large-scale results across multiple states. This means several different assessments show positive outcomes--an indication that the program is affecting something deeper than test prep for a specific type of assessment.

5. What outcomes were measured, and for which students? For a program to "stick" you should look for gains in standardized assessments or other aspects of an accepted accountability system. Also, the research should disaggregate improvements by initial proficiency levels, so you'd know whether there was improvement among both low and high-performers.

6. Is the program implementation replicable and sustainable? The study should identify conditions of implementation that led to results, and you should evaluate whether these conditions--training, time, acceptance by teachers, and cost--are replicable and sustainable for you.

Andrew R. Coulson is president of MIND Research Institute's Education Division.

COPYRIGHT 2012 Professional Media Group LLC
No portion of this article can be reproduced without the express written permission from the copyright holder.
Copyright 2012 Gale, Cengage Learning. All rights reserved.

Article Details
Printer friendly Cite/link Email Feedback
Title Annotation:SPONSORED SUPPLEMENT; research on K-12 curricular programs
Author:Coulson, Andrew R.
Publication:District Administration
Geographic Code:1USA
Date:Mar 1, 2012
Previous Article:What's wrong with math?
Next Article:Penguin rescues Colorado school district from flat math scores: Mind Research's ST Math brings excitement back to the numbers game.

Terms of use | Copyright © 2017 Farlex, Inc. | Feedback | For webmasters