Printer Friendly

A Powerful, Potential Outcomes Method for Estimating Any Estimand across Multiple Groups.

To read the full text of this article, click here:

Author(s): Pattanayak, Cassandra W.; Rubin, Donald B.; Zell, Elizabeth R.

In educational research, outcome measures are often estimated across separate studies or across schools, districts, or other subgroups to assess the overall causal effect of an active treatment versus a control treatment. Students may be partitioned into such strata or blocks by experimental design, or separated into studies within a meta-analysis. In non-randomized studies, students may be partitioned into subclasses based on key covariates or estimated propensity scores to improve observed covariate balance across treatment groups (e.g., Rosenbaum & Rubin, 1983). Procedures designed to estimate any estimand in the presence of strata, including a simple t-test for the difference in mean outcomes (Neyman, Iwaszkiewicz, & Kolodziejczyk, 1935), rely on implicit assumptions about the unknowable correlation between potential outcomes under active treatment and control treatment. For binary outcomes, the standard procedures used to estimate overall odds ratios in the presence of strata were introduced by Cochran (1954), who first proposed a hypothesis test for the difference in proportions across strata. Mantel and Haenszel (1959) proposed a very similar test and introduced an estimator for a common odds ratio. Consider the following hypothetical studies designed to estimate the causal effect of an existing program on high school graduation: (1) Within each of several school districts, half of the schools are randomized to participate in the program, and half are randomized not to participate in the program. (2) Within each of several cities, schools participating in the program are compared to schools not participating in the program, though participation was not randomized. (3) Several separate evaluations of the program are collected, to be combined in a meta-analysis. In each hypothetical study, a binary outcome (graduation) must be measured over strata (1. school districts, 2. cities, 3. evaluations). The effect of the program on graduation rates may be measured by a difference in proportions, odds ratio, or some other quantity. A hypothesis test will be conducted and a confidence interval constructed for the chosen estimand. The authors propose tests and intervals that can be more powerful for any finite population estimand than traditional tests and intervals, including t-tests for the difference in means or Cochran-Mantel- Haenszel procedures for the odds ratio.

ERIC Descriptors: Computation; Outcome Measures; Statistical Analysis; Graduation; Outcomes of Education; Influences; Simulation; Bayesian Statistics

Publisher: Society for Research on Educational Effectiveness. 2040 Sheridan Road, Evanston, IL 60208. Tel: 202-495-0920; Fax: 202-640-4401; e-mail:; Web site:

Source: Society for Research on Educational Effectiveness

ERIC Number: ED563037

Peer reviewed

Record Type: Non-Journal

Pages: 6

Abstractor: ERIC

Publication Type: Reports - Research

COPYRIGHT 2013 U.S. Department of Education
No portion of this article can be reproduced without the express written permission from the copyright holder.
Copyright 2013 Gale, Cengage Learning. All rights reserved.

Article Details
Printer friendly Cite/link Email Feedback
Author:Pattanayak, Cassandra W.; Rubin, Donald B.; Zell, Elizabeth R.
Publication:ERIC: Reports
Date:Jan 1, 2013
Previous Article:Making Sense of the "Zero Effect" of Comprehensive Teacher Induction Programs: A Mediational Analysis.
Next Article:Accounting for One-Group Clustering in Effect-Size Estimation.

Terms of use | Privacy policy | Copyright © 2019 Farlex, Inc. | Feedback | For webmasters