Flipping the research coin. (Research Corner: EDUCATION DATA AND RESEARCH ANALYSIS FROM EDVANTIA.Randomized ran·dom·ize tr.v. ran·dom·ized, ran·dom·iz·ing, ran·dom·iz·es To make random in arrangement, especially in order to control the variables in an experiment. field trials are not the best or only way to address k all important research questions, but they are often described as the "gold standard." This month's column explores why researchers and NCLB NCLB No Child Left Behind (US education initiative) express enthusiasm for experimental research designs, why schools may hesitate to participate, and possible win-win solutions. The researcher's view A randomized field trial is an experimental research design used to estimate the effect of an intervention or "treatment" (e.g., a math program) on a particular outcome (e.g., math achievement). Such trials are randomized and controlled. Random selection refers to the way participants are selected from a defined population (e.g., high-poverty rural schools). Random assignment occurs when investigators use a random process like the toss of a coin to decide which participants are assigned to an "experimental" group (which gets the intervention) and which are assigned to a "control" group (which does not). An experiment is controlled when investigators regulate all factors and conditions that could possibly influence the outcome or confound con·found tr.v. con·found·ed, con·found·ing, con·founds 1. To cause to become confused or perplexed. See Synonyms at puzzle. 2. the results. Researchers are especially fond of randomized field trials when evaluating educational programs because they can increase the validity of their findings. For example, if the experimental group improves while the control group does not, the difference isn't so easily attributed to factors such as prior achievement or classroom conditions. The educator's view School and district leaders want rigorous research information to help them select programs. But here's the rub: randomized field trials can't be done without the participation of schools and teachers--and testing educational theories in the real world is challenging. For example, in some cases, the ideal unit of study is individual students in the same classroom, but it's not always possible to simultaneously implement treatment and control programs (e.g., using textbook vs. computer-based exercises) without disrupting classroom flow. If the unit of study is the classroom or the school, teachers and principals may perceive a control-group designation as a "sentence to reform purgatory purgatory (pûrg`ətôr'ē) [Lat.,=place of purging], in the teaching of the Roman Catholic Church, the state after death in which the soul destined for heaven is purified. ," even though researchers may find the "old way" works best. Also, researchers must get parental consent Parental consent laws (also known as parental involvement or parental notification laws) in some countries require that one or more parents consent to or be notified before their minor child can legally engage in certain activities. before involving children in a study, and parents may perceive a control-group designation as unfair--or an experimental-group designation as "using their child as a guinea pig guinea pig (gĭn`ē), domesticated form of the cavy, Cavia porcellus, a South American rodent. It is unrelated to the pig; the name may refer to its shrill squeal. ." Possible solutions How can random assignment be made feasible and palatable? What could be done to make educators more likely to consider serving as the control group for a year or more, based on a coin flip? Johns Hopkins University Johns Hopkins University, mainly at Baltimore, Md. Johns Hopkins in 1867 had a group of his associates incorporated as the trustees of a university and a hospital, endowing each with $3.5 million. Daniel C. researcher Robert Slavin Robert Slavin is a noted psychologist who studies educational and academic issues. He founded the Success for All reform program for primary and middle schools. He will lead the Institute for Effective Education at the University of York - this is an international, has proposed a solution called "paired awards." Districts wishing to try an intervention would apply for a grant and, at the same time, identify pairs of schools (or teachers within schools) willing to participate in research. Those randomly selected as controls would continue using their regular programs for one year but would then be eligible to adopt the treatment under reduced costs or other concessions. University of Memphis The University of Memphis is a public research university located in Memphis, Tennessee, United States, and is a flagship public research university of the Tennessee Board of Regents system. researcher Steve Ross recommends refining the paired-awards approach so that the treatment-control comparison lasts at least two years, citing the implementation dip: "Given new strategies to understand and put into practice, teachers may struggle for a while, with short-term negative effects on teaching and learning. It would seem ironic if increasing randomized field trial usage caused the effectiveness of many potentially valuable programs to be systematically underestimated." For citation of the references used in this article, go to www.districtadministration.com www.edvantia.org, 800-624-9120 Scientifically Based Research Scientifically based research or SBR is the required standard in professional development and the foundation of academic instruction under the guidelines of No Child Left Behind Act of 2001 (NCLB).[1] References 1. NCLB defines it as "research that involves the application of rigorous, systematic and objective procedures to obtain reliable and valid knowledge relevant to education activities and programs." Scientific research in education has four basic steps (observe, hypothesize hy·poth·e·size v. hy·poth·e·sized, hy·poth·e·siz·ing, hy·poth·e·siz·es v.tr. To assert as a hypothesis. v.intr. To form a hypothesis. , collect data/evidence, draw conclusions), and does the following: 1. Poses significant questions 2. Links to relevant theory 3. Uses valid tools 4. Rules out alternative explanations 5. Produces findings that can be replicated 6. Survives scrutiny Source: NCLB, 2002; Committee on Scientific Principles for Education Research, 2002 DEFINITIONS Experimental research studies are randomized and controlled. Quasi-experimental studies introduce some controls over extraneous ex·tra·ne·ous adj. 1. Not constituting a vital element or part. 2. Inessential or unrelated to the topic or matter at hand; irrelevant. See Synonyms at irrelevant. 3. factors but lack at least one of the two defining characteristics of a true experiment. Quasi-experiments may be used when practical or ethic concerns prevent researchers from using an experimental study. |
|
||||||||||||||||||
Printer friendly
Cite/link
Email
Feedback
Reader Opinion