The causal effects of participation in the American Economic Association Summer Minority Program.

1. Introduction

In 1974, the American Economic Association sponsored a Summer Minority Program (hereafter AEASMP). An impetus for the AEASMP was the underrepresentation of racial minorities among the ranks of Ph.D. economists, in addition to the possible shortfalls in academic preparation that, if not addressed, would reduce the likelihood of minorities successfully completing doctoral programs in economics. Thus, from the start and continuing up to the present, the AEASMP provides participants with a curriculum that introduces, and perhaps remediates, core quantitative skills and, in recent years, research skills, essential for completing doctoral programs in economics. (1)

Given the severe underrepresentation of black Ph.D. economists in the profession, as an intervention, it is conceivable that the AEASMP could have an effect of increasing the likelihood of black participants earning economics doctorates and securing employment as economics faculty. (2) This seems especially plausible if the AEASMP enhances the capabilities of participants and/or cultivates a serious interest in becoming a Ph.D. economist. Notwithstanding such possibilities, nothing is known on the extent to which the AEASMP has causal effects.

As an intervention, it is of interest to know if the AEASMP has causal effects on outcomes deemed favorable for minority economists. Many outcomes associated with the training of Ph.D. economists are those outcomes associated with placement in tenure-track faculty positions in particular types of academic institutions. Successfully obtaining tenure is also associated with factors such as scholarly productivity, success in securing research resources, and in the prestige associated with scholarly society affiliation. If indeed the AEASMP has causal effects that enhance participants' labor market prospects as academic economists, the AEASMP can be viewed as a successful labormarket intervention.

This article examines the causal effects of AEASMP participation for minority Ph.D. economists. Methodologically, the treatment effect is identified with an extension of Rubin's (1974) potential outcomes approach. In this framework, there are two outcomes: a treatment state and a control state, and the causal effect of the treatment is simply the difference between the two potential outcomes. Given the nonrandomized nature of the data, differences in attributes between AEASMP participants and nonparticipants could lead to biased estimates of the causal effect of participation.

Possible bias in estimated treatment effects attributed to differing attributes is, of course, the sample-selection problem. A conventional approach assumes that selection into the treatment is conditioned on both observable and unobservable characteristics, and the treatment effect is estimated with latent variable selection models (Heckman 1979). However, the influential analysis of Lalonde (1986) has cast doubts on the accuracy of such models in estimating treatment effects. In contrast, when selection into the treatment is assumed to be conditioned only on observable characteristics, propensity scoring methods have been demonstrated to replicate experimental benchmarks reasonably well (Dehejia and Wahba, 1998). Below, we use both propensity-score weighted and Heckit estimators to identify and estimate the treatment effects of AEASMP participation. Our estimates of the treatment effects reveal that AEASMP participation has causal and positive effects on research productivity and on the ability to obtain research resources. We find similar effects for both propensity-score weighted and Heckit estimates.

The remainder of this article is organized as follows. A historical overview of the AEASMP is provided in section 2. Section 3 discusses the data. In section 4, we discuss our methodology--the potential outcomes approach to identifying treatment effects. Given selection on observable characteristics, unobservable characteristics, or both, we motivate the use of propensity-score weighted and Heckit estimators of the treatment effects attributed to AEASMP participation. Section 5 contains the results, where we report on treatment parameter estimates for the average effect of treatment in the population and the average effect of treatment on the treated. The last section concludes.

2. History and Overview of the AEASMP

The genesis of the AEASMP began in 1969 in the wake of the social and political unrest of the 1960s. (3) Members of the Caucus of Black Economists--the forerunner of the National Economic Association--held conversations with the American Economic Association Executive Board about strategies and programs for black Americans that would enable them to become Ph.D. economists. The impetus for these conversations was plausibly motivated by the severe underrepresentation of blacks among the economics faculty in U.S colleges and universities. Data from the National Science Foundation (NSF) WebCaspar database indicate that, in 1969, no doctorates in economics were awarded to black Americans, and the same was true for the three prior years. (4) This trend continued until 1974, when four doctorates were awarded to black Americans, half of which originated from Howard University--an historically black college/university (HBCU). Such a pattern underscores the anemic pipeline of black Ph.D. economists--the phenomenon of underrepresentation that continues to date (Collins 2000).

The AEASMP began its official history in 1974, with the University of California-Berkeley as the first host institution. Over the subsequent 29 years, the AEASMP has been hosted by eight additional institutions: Northwestern University (1975-1979), Yale University (1980-1982), University of Wisconsin-Madison (1983-1985), Temple University (1986-1990), Stanford University (1991-1995), University of Texas-Austin (1996-2000), University of Colorado-Denver (2001-2003), and Duke University (2004-present). To date, almost 800 students have participated in the AEASMP. Through 2001, a total of 46 out of 720 former AEASMP participants have gone on to earn doctorates in economics, with some 60 still enrolled in doctoral programs. As of June 2002, participants from the Temple program earned the most doctorates (13), followed by Northwestern (12), Yale (10), Stanford (5), Wisconsin (5), and University of California-Berkeley (1). Twenty-six black American AEASMP alumni were economics faculty members at the end of the 2001-2002 academic year.

Historically, admission into the AEASMP has been restricted to applicants who have completed at least the sophomore year of college. Admission is also competitive, based primarily on the applicant's grade point average and, according to Leeds (1992), the selectivity of the applicant's undergraduate institution--given that the applicant is at least a sophomore. An exception to the admission criteria occurred when Temple University hosted the AEASMP. Concerned about a possible proskill bias in the admission criteria that would deny opportunities to the most socially disadvantaged racial minorities, the Temple AEASMP reduced both the role of grades and the selectivity of the applicant's undergraduate institution in admission decisions (Leeds 1992). Relative to cohorts that participated in the AEASMP, the Temple experiment does not appear to have had any adverse effects. Participants of the AEASMP at Temple account for the most economics doctorates earned by AEASMP alumni through the year 2001.

Throughout its history, the central goal of the AEASMP has been to prepare undergraduates from underrepresented racial and ethnic groups for doctoral studies in economics. Presumably, the preparation enabled by the AEASMP would increase the numbers of minorities in the economics profession--a goal of the AEASMP articulated by Leeds (1992). As such, the typical host of the AEASMP provides for participants over a period of approximately eight weeks in the summer, advanced coursework in microeconomics, macroeconomics, and quantitative methods--some combinations of mathematical economics and/or econometrics/statistics. Beginning with the tenure of the University of Colorado-Denver as host in 2001, the curriculum underwent three major innovations. Whereas previous hosts offered one uniform level of instruction, the AEASMP now admits participants in one of two tiers, with two levels of instruction (foundations and advanced). Advanced coursework in macroeconomics has also been replaced with coursework in research methodology and data analysis. It is also possible for an individual to participate in the AEASMP for two summers. Participants that are initially admitted into and complete the foundations part of the program are also eligible to come back as participants in the advanced part of the program.

Follow-up survey data collected by the AEASMP suggest that it has been reasonably effective at increasing the number of minorities on the economics faculties of colleges/universities. Through the 2000-2001 academic year, some 26 black American and 2 Hispanic AEASMP alumni were economics faculty, which includes at least three faculty in top-ranked economics programs. The AEASMP may have effects on the number of minorities in other professions as well. Follow-up survey data also indicate that, through 2001, some 77 AEASMP alumni earned a Juris doctorate or some other type of doctorate. Of course, such retrospective outcomes do not imply causality, as we cannot observe the counter factual history of an individual had he/she not been an AEASMP participant. Neither do such data allow an inference as to whether or not AEASMP participation effected some capability necessary for success in a given profession--economics faculty or otherwise.

3. Data

Broadly construed, a treatment effect of program participation is simply an induced change, after participation in a program, in some outcomes. These outcomes can be either be associated with, or interpreted as, success. For individuals with economics doctorates, beyond earning the doctorate itself, success can also be measured along other dimensions. One measure is the type and quality of the university where one obtains employment. Given one's employment at a college/university of a particular type, promotion/tenure and intellectual status will be determined by research productivity, membership in prestigious scholarly societies, and in demonstrated ability to obtain research resources. While there are perhaps many possible treatment effects of AEASMP participation--some of which were not even a goal in program design, we focus on those that we can observe, measure, and are likely to be associated with successful careers as economics faculty. As such, we explore whether or not, for black American AEASMP alumni who earned economics doctorates, AEASMP participation causally affected the type of university where employed, research productivity, placement in scholarly organizations, and ability to attract research resources.

To assess whether or not the AEASMP has treatment effects, our sample of treated individuals consists of black American AEASMP alumni who were economics faculty members at a U.S. college/ university as of the 2000-2001 academic year. While AEASMP participation is not restricted to black Americans and there are two Hispanic AEASMP alumni on economics faculty as of the 2000-2001 academic year, we restrict the sample of treated individuals to black Americans for several reasons. Through the 2000-2001 academic year, approximately 93% of the AEASMP alumni on economics faculty were black American. Such a sample restriction does not mean that the possible treatment effects of AEASMP participation are limited to black Americans. It merely reflects that, through the 2000-2001 academic year, nonblack participants of the AEASMP have yet to show up in significant numbers on economics faculties. In recent years, the AEASMP has had a significant number of nonblacks as participants, including Native Americans, and economic and socially disadvantaged Southeast Asians. As such, there will undoubtedly be significant numbers of nonblack participants that show up on economics faculty as the AEASMP continues its history.

Restricting the sample of treated individuals to black Americans is also a practical necessity, as we cannot think of a sensible and cost-effective way to construct a control group of Hispanics with economics doctorates. Historically, Hispanic eligibility to the AEASMP was restricted to so-called economic and socially disadvantaged and underrepresented Hispanics, such as Puerto Ricans and Chicanos. While members of such groups typically have a Spanish surname, finding a control group based on Spanish surname would not only identify other economic and socially disadvantaged Hispanics, but Hispanics for whom the official designation of economic and socially disadvantaged is not applicable. For example, an individual on an economics faculty could have a Spanish surname, but neither be Puerto Rican nor Chicano. Instead, he could be a migrated elite or a descendant of a family of migrated elites, originating from countries such as Argentina, Colombia, Cuba, or Spain.

Identification of black AEASMP alumni on economics faculty was obtained by matching names from alumni lists kept by the AEASMP to the Prentice Hall Economics Faculty Guide (Hasselback 2000). This procedure allowed us to identify those black AEASMP alumni who had earned economics doctorates and were employed as an economic faculty member in a U.S. college/university during the 2000-2001 academic year. In cases where a name and college/university affiliation was on the AEASMP alumni list but did not show up in the Faculty Guide, verification was obtained through a college/university website.

Our control group consists of black economists who were employed in a college/university at some point in the 2000-2001 academic year. For purposes of sample inclusion, an economist is considered black if he or she is a non-Hispanic black American or black African. Several sources were utilized to obtain control-group observations on black economists. First, we identified all black economists listed in the 1988 Directory of Black Economists compiled by Burbridge and Jones (1989) for the National Economic Association. Those individuals listed in the directory were then cross-checked with the Prentice Hall Economics Faculty Guide 2000/2001 to determine if they were still employed as an economics faculty member at a college/university at some point in the 2000-2001 academic year. An additional source of observations was from the personal knowledge of individuals about where black economists are employed. Based on the knowledge of the author of this paper and that of several colleagues, additional names of black economists employed in colleges/universities as of the 2000-2001 academic year were obtained and added to the observations generated from the first two sources. All three sources generated a total of 180 black Ph.D. economists.

To measure the research productivity of black economists, articles, working papers, books, and book chapters listed in the EconLit database attributed to black economists in the sample dated no later than December of 2000 were obtained. (5) This database covers approximately 1100 journals, most of which are economics journals proper, and form what constitutes the economics literature. (6)

All constructed measures of research output are based on counts, and no adjustment for coauthorship is made. While coauthorship may be important, we ignore a popular convention in the literature of assigning a weight of 1/n to coauthored publications, where n is the number of co-authors, and exploit the natural discreteness of article publications, which inform count data-based models of economics faculty research productivity as in Agesa, Granger, and Price (2000). We also refrain from adjusting publications for quality on the basis of equivalance weights as in Laband and Piette (1994). This allows us to capture black Ph.D. economist research productivity across the entire range of the economics literature, as measured in the EconLit database. (7)

While we do not adjust for the quality of publications by utilizing standardized citation-based weights, we do follow the convention of Scott and Mitias (1996) and report counts of articles in the top economics journals. We add to the 36 considered by Scott and Mitias the Journal of Economic Perspectives, given its fast rise to the top of the economics hierarchy, as described by Laband and Piette (1994). (8)

To measure the type of college/university where a black Ph.D. economist is employed, a binary variable is added according to three Carnegie classifications: Research I or II, Liberal Arts I or II, and Selective Liberal Arts. Carnegie classifications were obtained from the NSF WebCASPAR database.

A variable measuring access to funded research resources was constructed on the basis of whether or a not an individual black Ph.D. economist had ever received a NSF economics research grant. This confidential information was obtained from internal administrative NSF data and is recorded as a binary variable.

The scholarly affiliations of black Ph.D. economists were determined on the basis of whether or not they are associates, research or faculty, of the National Bureau of Economic Research (NBER) as of December of 2002. It is a binary variable and is based on the NBER roster of associates available on-line. (9)

4. Methodology

Viewed as an intervention designed to engender favorable outcomes for participants, the AEASMP is subject to the so-called evaluation problem (Schmidt and Kluve 2001). In general, it is impossible to observe for participants a counterfactual post-AEASMP history in which they did not participate. In nonexperimental restropective data, this problem is particularly vexing, as, in a sample of both participants and nonparticipants, there may be omitted nonrandom factors that systematically condition behavior and selection into the program--the so-called sample-selection problem.

Given the possibility of nonrandom selection into treatment, there are two broad approaches to estimating the effects of treatment. A first approach attempts to explicitly model the selection process. If selection into the treatment is conditional on a set of observable characteristics, independent of unobservable characteristics, propensity score methods (Rosenbaum and Rubin 1984) are appropriate. If, however, selection into the treatment depends on both observable and unobservable individual characteristics, the latent variable selection approach of Heckman (1979) is appropriate. Both of these approaches are based on untestable assumptions.

A second approach avoids modeling the selection process and instead considers the distributions of the treatment and outcome(s), allowing one to construct a range of estimates, or bounds, consistent with the observed data (Manski 1990). These bounds can be tightened by imposing untestable assumptions. Lechner (1999), for example, achieves tighter bounds on treatment effects by assuming that selection into treatment is conditioned on individuals with positive expected returns to treatment. To the extent the estimated bounds include zero, Lechner (1999) shows how the estimates can be bounded away from zero with appropriate exclusion restrictions.

We adopt the first approach for estimating the treatment effects of AEASMP participation. Bound estimates of treatment effects require sufficiently large samples (Lechner 1999) in order to obtain unbiased nonparametric kernel estimates of bootstrapped sampling distributions, and our sample is small. (10) Even in sufficiently large samples, there is no consensus on the asymptotic properties of nonparametric bound estimators. As such, any inferences in a small sample about the true magnitude of treatment effects within estimated bounds is subject to uncertainty and do not offer anything precise about the effects of treatment that are likely to be meaningful for assessing the impact of a program like the AEASMP. Lechner (1999), in his evaluation of treatment bounds utilizing the method suggested by Manski et al. (1992), concludes that the use of bounds as a test for predictions of parametric selection models have low power. This suggests that, while parametric selection models are based on perhaps false assumptions about how participants select into the treatment, nonparametric bound estimates of treatment effects do not offer a decisive and unambiguous alternative.

Our empirical analysis of AEASMP treatment effects follows the potential outcome approach of Rubin (1974). For each unit in a sample from a population, T measures whether treatment was received, where T = 1 if treatment was received and zero otherwise. The variables Y(1) and Y(0) are the outcomes associated with treatment and control, respectively, with the treatment being having participated in the AEASMP. The two treatment effects of interest are

[tau] = E[Y(1) - Y(0)]

[[tau].sub.t] = E[Y(1) - V(0) | T = 1],

where E is the expectation operator and [tau] and [[tau].sub.t] are measures of the average effect of treatment on the population and treated population, respectively.

If assignment to treatment and the responses Y(1) and Y(0) are independent conditional on a vector of pretreatment variables x, then, for individuals defined by values of these pretreatment variables, there is random assignment into the treatment. This is the assumption of unconfoundedness (Rosenbaum and Rubin 1983, 1984), and it permits one to view the observations as being randomly assigned to treatment, given observed pretreatment variables. (11)

To identify and estimate the effects of the treatment, we first implement a propensity-score method. For a vector of variables x that are unaffected by treatment, a propensity score, which serves as an assignment indicator as to whether or not treatment occurs, is

e(x) = Prob(T= 1 | x),

where 0 < e(x) < 1. Given the unconfoundedness assumption, treated and control observations with equal or nearly equal propensity scores will tend to have pretreatment covariates that are similar, and their allocation, actual or hypothetical, to the treatment or control group can be regarded as if random. (12) Thus, conditional on the propensity score, treatment assignment is independent of the responses Y(1) and Y(0).

It follows that an unbiased estimator of the treatment effects is based on

[tau](x) = E[Y(1) - r(0) | x, e(x)].

Thus, if we adjust, in some manner, the treated and control observations for the propensity score, we can make valid and unbiased causal inferences about the effect of treatment--participation in the AEASMP.

Given selection into treatment that is a function of observable characteristics, our estimator of the treatment parameter is

E[Y | T = t, e(x)] = [[beta].sub.o] + [n.summation over (I=1)] [[beta].sub.i][V.sub.i] + [tau] x [[T.sub.i] + [[epsilon].sub.i], (1)

where the [V.sub.i] are control variables and [[epsilon].sub.i] is an error term. The interpretation of the treatment effect given by the estimate of z in Equation 1 depends on the choice of weights, [omega](t, e(x)), and we consider the following:

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] (2)

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] (3)

The estimate of z in Equation 1 weighted by Equation 2 measures the average treatment effect for an individual randomly drawn from the population. When the estimator is weighted by Equation 3, the estimate of [tau] measures the average treatment effect for an individual randomly drawn from the population of training participants, or the effect of treatment on the treated. This estimate of [tau] is of particular interest, as it measures the effect of treatment on the population that is most likely to participate in the treatment.

The weights in Equations 2 and 3 require some estimate of the propensity score that is a function of a set X of pretreatment covariates. We follow the convention of basing the estimate of the propensity scores on the predicted values from a logistic regression model,

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

where the [x.sub.i] are pretreatment covariates.

The estimator in Equation 1, with weights given by Equation 2 or 3, is similar to that of Hirano, Imbens, and Ridder (2000), who show that weighting a regression by the inverse of a propensity score estimated from a nonparametric series estimator (Newey 1994) renders the regression into an empirical likelihood estimator that efficiently incorporates information about selection into treatment. It is also similar to the estimator proposed by Horvitz and Thompson (1952) that accounts for sample items with unequal probabilities of selection. As our sample size is too small to implement a nonparametric series-type estimator of the propensity score suggested by Hirano, Imbens, and Ridder (2000), while inefficient, our estimates of the treatment effects will be unbiased. (13) If selection into the treatment is independent of unobserved characteristics, the estimate of [tau] in Equation 1 weighted by the relevant propensity score is an unbiased estimate of the causal effect of the treatment. This assumption is untestable, and if unobserved characteristics also matter for selecting into the treatment, the estimate of [tau] will be biased.

Given the possibility that unobserved characteristics matter for selection into the treatment, we also identify and estimate the treatment effects of AEASMP participation with a standard two-step Heckit estimator. In this context, selection into the treatment T is modeled as a latent variable [T.sup.*.sub.i] = [[summation of].sup.n.sub.i=0] [[alpha].sub.i][x.sub.i] + [[mu].sub.i] were, for T [member of] [0, 1], the [x.sub.i] are observed characteristics and [[mu].sub.i] is an error term measuring unobserved characteristics. (14) The standard Heckit estimator of treatment effects proceeds by recognizing the correlation between the errors in Equation 1 and the latent variable selection process [T.sup.*], or E([[mu].sub.i] | T [member of] [0, 1]). If the error terms [[epsilon].sub.i] and [[mu].sub.i] are jointly normal in distribution, this conditional expectation becomes an inverse Mills ratio--the selectivity correction to be included in an ordinary least squares (OLS) regression of a specification such as Equation 1.

5. Results

Table 1 reports on the mean and standard deviation of relevant variables for the entire sample, the control group, and the treatment group. For the 180 black economists in the sample, the following variables were constructed: a binary variable indicating whether or not the individual is a native-born black American (BAMER), a binary variable indicating whether or not the individual earned an undergraduate degree from an historically black college/university (HBCUBA), the year in which the economics doctorate was earned (YRPHD), total number of published JEL-indexed articles (TJEL), total number of JEL-indexed articles in the top 37 journals (TOP37), three binary variables indicating whether or not the individual is employed at a research institution (RESU), liberal arts institution (LARTS), and selective liberal arts institution (SLARTS), a binary variable indicating whether or not the individual is an associate of the NBER, and a binary variable indicating whether or not the individual has ever received a research grant from the National Science Foundation (NSFSUP). (15) We also create, to control for possible differences in the quality of AEASMP training, binary variables indicating whether or not an individual attended the AEASMP at Stanford University (STAN), Temple University (TEMP), University of Wisconsin (WISC), or Yale University (YALE).

Table 1 reveals that approximately 77% of AEASMP participants were black Americans over the sample period under consideration. This suggests that any significant estimates of treatment effects, especially the effects of treatment on the treated, will be most relevant for, and interpreted as applying to, black Ph.D. economists. The sample means of the variables also reveal additional differences between the control group and the treated group--participants of the AEASMP relative to the control group, members of the treated group are less likely to have earned their baccalaureate degree from an historically Black college/university and more likely to have an economics doctorate that is of recent vintage. The means of the research productivity measures suggest that AEASMP participants are less productive than nonparticipants. Finally, relative to the control group, members of the treated group appear more likely to be employed in research, liberal arts, and selective liberal arts institutions, more likely to have an NBER affiliation, and more likely to have received research funding from the NSF. With regard to the institution where participants received the treatment, for the time period covered by the sample, Yale University hosted the largest number of AEASMP participants.

Table 2 reports for the full sample, parameter estimates of the average treatment effects and the effect of treatment on the treated. When the outcome variable is binary (LARTS, SLARTS, NBER, NSFSUP), the estimate of [tau] measures the effect of treatment on the linear probability of success. The parameter estimates in columns 1 and 2 are propensity-score weighted estimates of the average treatment effect (ATE), and the effect of treatment on the treated (TTE), respectively. Columns 3 and 4 report corresponding Heckit parameter estimates, along with the inverse Mills ratio for the TTE. (16) Each column reports on the estimate of [tau]--the treatment effect of AEASMP participation, for the outcome variable in the first column. For the Heckit specifications, the selection equation has a specification equal to the specification for the estimated propensity score described below. For all specifications, we allow for cohort effects by adding as controls (the [V.sub.i] from Eqn. 1) dummy variables for the host institution where the participant received the treatment. Parameter estimates of the effects of the control variables are not reported for the sake of brevity.

The estimated propensity score is based on the following specification:

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

where YRPHD is used as a proxy for the age of the individual. All three of these covariates are sensible candidates for determining the probability of selection into the AEASMP, as they are not capable of being altered by the treatment itself. The inclusion of the interaction terms also provides some flexibility to the propensity score as a general specification of the probability of actually receiving the treatment.

The propensity-score weighted parameter estimates in Table 2 suggest that, if selection into the treatment is based on only observable characteristics, AEASMP participation has five causal and positive impacts. For both an individual drawn randomly from the population and those actually receiving the treatment, AEASMP participation has a causal and positive impact on publishing in top economics journals, and in securing research resources from the NSF. With respect to obtaining membership in the NBER, AEASMP participation only has a causal and positive impact on those that actually received the treatment. If selection into the treatment is based on both observable and unobservable characteristics, the Heckit parameter estimates in columns 3-4 suggest that AEASMP participation only causally affects the securing of NSF research resources for an individual randomly drawn from the population. The insignificance of [lambda] in the Heckit specification for the effects of treatment on the treated in column 4 also suggests that participation in the AEASMP is not subject to selection bias.

Of the seven outcome variables under consideration, the results in Table 2 suggest that AEASMP participation only has a causal impact on three outcomes when weighting by the propensity score. For the corresponding Heckit estimates, only one outcome is causally affected by AEASMP participation. Even though the propensity score balances the distribution of covariates between the control and treatment observations, it still could be the case that, across the distribution of both the control and treated observations, comparability is not exact enough to infer robust treatment effects. The Heckit parameter estimates could also be compromised in this context. Even though selection into the treatment in Heckit models is based on both observables and unobservables, large unobserved differences in characteristics between the treated and control units could conceivably introduce some unobserved heterogeneity and downward bias in parameter estimates. In consideration of this, Tables 3-5 report on treatment effects under various truncations of the propensity score.

Table 3 reports treatment parameter estimates after the sample is truncated by deleting those control units with propensity scores lower than the minimum propensity score among the treated units. (17) Given this strategy of making the control units more similar to the treated units by truncation, the parameter estimates in Table 3 are approximate to the estimates in Table 2--AEASMP participation has four causal effects. The exception is that, for membership in the NBER, AEASMP participation no longer has a causal effect when weighting by the propensity score. The Heckit parameter estimates remain the same, with no evidence of selectivity bias.

Table 4 reports treatment parameter estimates after the sample has been truncated by deleting those control observations with propensity scores less than the average propensity score for the treated observations. (18) The results in Table 4 suggest that, if selection into the AEASMP is based on observable characteristics, AEASMP participation has only two causal and positive effects--total publications for a randomly drawn individual from the population and securing research resources from the NSF for individuals actually receiving the treatment, lf selection is based on both observable and unobservable characteristics, the Heckit parameter estimates suggest that there is only an average treatment effect for securing NSF resources, and when the outcome is publishing in top economics journals, there is no causal effect but, given the significance of [lambda], there is selection into the treatment.

Table 5 reports estimates after the sample has been truncated by deleting those control observations with propensity scores less than the percent of black economists in the sample that actually received the treatment. (19) For selection into the treatment based on observable characteristics, and given the sample truncation, the propensity-score weighted parameter estimates suggest that AEASMP participation has a causal effect on three outcomes: total publications, publications in top journals, and securing NSF support. The causal effect on total publications is for both an individual randomly drawn into treatment and for an individual receiving the treatment. The Heckit parameter estimates are similar to what they are in Table 4 and remain dissimilar to the propensity-score weighted estimates.

Two general observations can be made about the treatment-effect parameter estimates reported in Tables 2-5. First, they reveal the sensitivity of propensity-score weighted estimates of the two types of treatment effects under consideration to the matched control cohort. Whether or not AEASMP has a causal effect on some outcome variable for a randomly drawn member of the population or for an individual actually receiving the treatment varies with the propensity-score-based sample truncation. The outcome variable measuring NSF support is always positive and significant, suggesting that the ability of black economists to obtain NSF research resources is causally and positively related to AEASMP participation. At least one of the publication variables is also robust with respect to the sample truncations. This suggests that black economist research productivity is causally and positively related to AEASMP participation.

A second general observation about the results in Tables 2-5 is that the Heckit estimates of the two types of treatment effects differ dramatically from the propensity-score weighted estimates. In general, it appears that assuming that selection into the treatment is based solely on observable characteristics versus both observable and unobservable characteristics matters for the effects of a treatment. Neither of these assumptions is testable. Differences between the propensity score and Heckit treatment effect estimates may also result in part from the the fact that the distribution underlying selection into treatment is also different for the propensity score and Heckit treatmentparameter estimates. The propensity score is estimated via Logit, whereas for the Heckit model, selection into treatment is estimated via Probit. While both Probit and Logit are similar, if there are a significant number of observations at the tails of a distribution, a Logit specification is preferable and can result in different parameter estimates than a Probit. If, for example, the sample contains a significant number of individuals who have extremely low and extremely high probabilities of selecting into the AEASMP, a Probit specification of selection into treatment may not be appropriate.

In Table 6, we examine the sensitivity of the Heckit parameter estimates to the specification of the probability distribution for selection into the treatment. We report Heckit treatment-effect estimates across the samples considered in Tables 2-5, with the cumulative logistic probability function determining selection into the treatment. For sake of brevity, only the effects of treatment on the treated are reported. The results reveal that, in contrast with the previous Heckit estimates, AEASMP participation always has a positive and causal effect on securing NSF research resources, and in the majority of the specifications, on publishing in top economics journals. With regard to selection bias, [lambda] is significant in five cases where a treatment effect is present. The Heckit estimates of the AEASMP treatment effects pick up more of those identified in the propensity-score weighted specifications when selection in the Heckit model is the same probability distribution as the propensity score--the cumulative logistic.

The empirical relevance of the treatment effects identified in Tables 2-6 are sensitive to the specification being either propensity-score weighted or based on the Heckit method--or depend on whether selection into the AEASMP is based just on observable characteristics, or both observable and unobservable characteristics. Pragmatically, the different estimates provided by the propensity-score weighted and Heckit model suggest a range of values for treatment effects. In general, both specifications commonly identified, across various propensity-score truncated sample sizes, AEASMP participation as having a positive and causal impact on research productivity in top economics journals and in success securing NSF research support. The effects of treatment on the treated are of particular interest, as the evaluation of any program must consider the impact it had on actual participants and not its effects on an individual drawn randomly from the population, or average treatment effects.

Our own view is that the most compelling estimates of the treatment effect of AEASMP participation are those in Table 3 for the propensity-score weighted specification and the second column of Table 6 for the Heckit specification. Restricting the controls to those individuals with a probability of selection equal to or greater than that of treated individuals seems sensible, as control group individuals with lower probabilities of selection are not likely to be candidates for participation. As such, their presence in the sample could impart a downward bias of the effect of treatment on the treated, as the results in Table 2 and the second column of Table 6 suggest.

If selection into the AEASMP is determined only by observable characteristics, the results in column 2 of Table 3 indicate that participation in the AEASMP causes research in top economics journals to increase, relative to nonparticipants, by 0.873 articles. AEASMP participation also increases by 0.479 the probability of securing research support from the NSF. If selection into the AEASMP is determined by both observable and unobservable characteristics, AEASMP participation has an even larger effect on these two outcomes. The results in the second column of Table 6 indicate that participation in the AEASMP causes research in top economics journals to increase, relative to nonparticipants, by 1.92 articles. AEASMP participation also increases by 0.621 the probability of securing research support from the NSF.

The magnitudes of the causal effects of AEASMP participation, while seemingly small, are not necessarily trivial. The analyses of Agesa, Granger and Price (1998, 2000) indicate that the number of economics doctorates earned by black Americans is positively related to, and the probability increases with respect to, the research productivity of black economics faculty. Thus, increases in the research productivity of black Ph.D. economists, especially in top economics journals, besides increasing their prospects for tenure, also appear to have a pipeline effect--inducing black baccalaureate graduates to earn doctorates in economics. In this context, the direct causal and positive impact of AEASMP participation has a direct effect of making participants more research productive and an indirect effect of enabling and feeding a pipeline of future black Ph.D. economists. (20)

With regard to the effects of AEASMP participation on securing research resources from the NSF, the estimated effects, while seemingly small, are potentially quite substantial. Feinberg and Price (2004) show that, while the probability of obtaining NSF research support in economics is approximately 33%, accounting for social capital, particularly membership in NBER, reduces the odds of success substantially for the typical NSF grant applicant. Moreover, the success rate of nonwhite minorities is approximately 4%. As such, the results reported here suggest that participation in the AEASMP increases the likelihood, relative to nonparticipants, of securing NSF research resources. If, as Wachtel (2000) argues, that obtaining an NSF grant for economics research also increases success probabilities associated with article acceptances, obtaining grants from other agencies/foundations, and in overall career advancement, the estimated causal effects of AEASMP participation on securing NSF research support induce larger and indirect successful outcomes.

6. Conclusions

Although the AEASMP has been in existence for approximately three decades, nothing is known regarding the causal effects, if any, associated with participation in the program. This article has provided evidence that there are some favorable and nontrivial treatment effects of AEASMP participation. Utilizing propensity-score weighted and standard Heckit estimates of treatment effects on a sample, where the treated group is majority black American, the results reported here suggest that AEASMP participation by black American Ph.D. economists has a positive and causal impact on four outcomes associated with success as an academic economist: (1) publications in all JEL-indexed journals, (2) publications in top 37 JEL-indexed journals, (3) membership in NBER, and (4) success in securing economics research resources from the NSF. While the estimates of the treatment effects are sensitive to and differ across the propensity-score weighted and Heckit specifications of the potential outcomes associated with participating in the AEASMP, the effects of treatment are similar in pattern and magnitude when the sample is restricted to controls for which the propensity score is restricted to being at least as large as those who received the treatment--the results in column 2 of Table 4 and the second column of Table 6. In our view, this is a sensible selection process, and the associated sample truncation results in a sample of individuals with similar unobservable characteristics conditioned on the propensity score. As such, our results suggest there are robust treatment effects, for both ATE and TTE, with respect to publishing in top economics journals and securing research resources from the NSF.

Given the highly selective admissions criteria of the AEASMP over most of the sample period under consideration, one would expect the effects of AEASMP participation to be quite small or perhaps even nonexistent. Our outcome measures are clearly a function, in part or whole, of unobserved ability. It is plausible that the AEASMP, by virtue of its admission criteria, provides instruction/training to individuals with high ability, who may have had favorable outcomes as academic economists had they not received the treatment. Our estimates of the effects of AEASMP participation account for such selection bias and suggest that the AEASMP has treatment effects that are neither small or nonexistent.

Viewed as a labor-market intervention targeted at minority groups underrepresented in the economics profession, our results suggest that the AEASMP is both directly and indirectly an effective and successful strategy for increasing the representation of minority groups in the economics professoriate. As publishing in top economics journals and securing research resources are important for promotion and tenure, our results suggest that the AEASMP has a direct effect on diversifying economics faculty at the tenured and senior ranks. Given the positive effects that the research productivity of black economists has on black undergraduates that go on to earn economics doctorates (Agesa, Granger, and Price 2000), our results also suggest that the AEASMP can have an indirect effect on the pipeline of future black Ph.D. economists.

There are some possible limitations of the results reported here. If there are omitted variables in our regression specifications that are important for explaining the outcomes under consideration, the estimates of the treatment effects could be biased. In addition, the estimates of the treatment effects of AEASMP participation may understate the causal impacts of the program for two reasons. First, it could be the case that the propensity-score weighted estimates are spuriously imprecise as a result of the control and treated units not having sufficient overlap in their propensity scores (Imbens 2004). If so, some of the treatment effects considered, but found to be insignificant, may indeed be significant. Second, there could be other outcomes important for success as an academic economist other than the seven we consider on which the AEASMP could have a causal and favorable impact.

The seven outcomes under consideration are, in our view, partly a function of ability, which is unobserved. This clearly would introduce a source of unobserved heterogeneity as ability is not observed, and we can think of no valid instrument for it. Unobserved ability could also be correlated with the decision to select into the treatment and with other variables that determine the outcomes under consideration. However, the similarity of the magnitudes of the estimated treatment effects between the propensity-score weighted specification in the second column of Table 3 and the Heckit Logit selection specifications in the second column of Table 6 suggest that, if there is omitted variable bias, it is either small or nonexistent. The Heckit specification accounts for unobservable characteristics, which includes ability, and other individual characteristics correlated with ability. As such, the similarity between the Heckit and propensity-score weighted estimates of the AEASMP treatment effects suggests that omitted variable bias is either small or nonexistent.

Valid inference with propensity-score weighting requires sufficient overlap in the distribution of propensity scores among treatment and control groups. Constructing such distributions requires nonparametric kernel or series estimators, which require large samples to minimize bias. As our sample is small, we cannot be sure that AEASMP participation does not causally affect any of the other outcomes we considered. However, to the extent that a lack of overlap in the distribution of propensity scores between the treated and controls results in large standard errors for the parameter estimates (Imbens 2004), our results are compelling. The standard errors of the estimates do not differ substantially in magnitude across the propensity-score weighted and Heckit estimates. This suggests that there is sufficient overlap in the propensity score to enable valid inference on the treatment effects when selection is conditional on observable characteristics.

Finally, the AEASMP may have a causal and favorable impact on other outcomes, not considered in this paper, that are important for success as an academic Ph.D. economist. For example, part of the underrepresentation of black Americans among the ranks of Ph.D. economics faculty may be explained by a constrained flow of black Americans into economics doctoral programs. The AEASMP may have a causal effect on black American entry into economics doctoral programs. However, establishing a control group to examine this is not practical, and such a causal effect of AEASMP participation would be important if it exists. Presumably, one must first attend and finish an economics doctoral program in order to publish in top economics journals and successfully compete for NSF research resources.

Table 1. Sample Means and Standard Deviations

Variable       Sample       Control Group   Treatment Group

BAMER      0.544 (0.499)    0.506 (0.501)    0.769 (0.429)
HBCUBA     0.183 (0.388)    0.195 (0.397)    0.115 (0.326)
YRPHD      1985.72 (8.89)   1984.78 (9.02)  1991.31 (5.53)
TJEL        5.39 (8.89)      5.71 (9.47)      3.54 (3.48)
TOP37       1.47 (3.18)      1.53 (3.38)      1.11 (1.56)
RESU       0.367 (0.483)    0.351(0.479)     0.461 (0.508)
LARTS      0.144 (0.352)    0.143 (0.351)    0.154 (0.368)
SLARTS     0.122 (0.328)    0.110 (0.314)    0.192 (0.402)
NBER       0.033 (0.180)    0.026 (0.159)    0.077 (0.272)
NSFSUP     0.128 (0.335)    0.097 (0.297)    0.308 (0.471)
STAN       0.167 (0.128)          -          0.115 (0.326)
TEMP       0.277 (0.165)          -          0.192 (0.401)
WISC       0.167 (0.128)          -          0.115 (0.326)
YALE       0.333 (0.180)          -          0.231 (0.429)
N               180              154              26

Standard deviations are reported in parentheses.

N = number of observations.

Table 2. Full Sample Estimates of Treatment Effects

Specification             (1)                 (2)

                   Propensity-Score    Propensity-Score
Outcome              Weighted ATE        Weighted TTE

TJEL                 0.787 (1.31)        1.23 (0.729)
[[lambda].sub.i]          --                  --

TOP37              1.04 (0.498) (b)    0.862 (0.299) (a)
[[lambda].sub.i]          --                  --

RESU                -0.104 (0.093)      -0.076 (0.104)
[[lambda].sub.i]          --                  --

LARTS               -0.070 (0.066)      -0.007 (0.073)
[[lambda].sub.i]          --                  --

SLARTS              -0.039 (0.063)       0.011 (0.074)
[[lambda].sub.i]          --                  --

NBER                 0.019 (0.033)     0.083 (0.044) (c)
[[lambda].sub.i]          --                  --

NSFSUP             0.279 (0.071) (a)   0.484 (0.070) (a)
[[lambda].sub.i]          --                  --

N                         180                 180

Specification
                          (3)                (4)

Outcome               Heckit ATE         Heckit TTE

TJEL                 -0.532 (2.91)      -4.26 (4.65)
[[lambda].sub.i]          --             2.68 (2.66)

TOP37                0.538 (1.04)       -0.249 (1.69)
[[lambda].sub.i]          --            0.568 (0.957)

RESU                -0.010 (0.165)      0.304 (0.266)
[[lambda].sub.i]          --           -0.227 (0.151)

LARTS               -0.035 (0.122)     -0.112 (0.197)
[[lambda].sub.i]          --            0.056 (0.112)

SLARTS              -0.004 (0.112)     -0.021 (0.181)
[[lambda].sub.i]          --            0.012 (0.103)

NBER                 0.083 (0.061)      0.101 (0.098)
[[lambda].sub.i]          --           -0.013 (0.056)

NSFSUP             0.459 (0.111) (a)    0.251 (0.178)
[[lambda].sub.i]          --            0.151 (0.101)

N                         180                180

Control variables include dummy variables indicating whether
the treatment was received at Stanford, Temple, Wisconsin, or Yale.

Standard errors are in parentheses.

N = number of observations.

(a) Significant at the .0l level.

(b) Significant at the .05 level.

(c) Significant at the .10 level.

Table 3. Propensity Score-Truncated Estimates
of Treatment Effects: e(x) < 0.0319

Specification            (1)                (2)

                   Propensity-Score   Propensity-Score
Outcome              Weighted ATE       Weighted TTE

TJEL                 1.07 (1.01)        1.32 (0.815)
[[lambda].sub.i]          --                 --

TOP37              1.31 (0.407) (a)   0.873 (0.345) (b)
[[lambda].sub.i]          --                 --

RESU                -0.032 (0.115)     -0.076 (0.124)
[[lambda].sub.i]          --                 --

LARTS               -0.078 (0.083)     -0.007 (0.088)
[[lambda].sub.i]          --                 --

SLARTS              -0.036 (0.081)     0.010 (0.090)
[[lambda].sub.i]          --                 --

NBER                0.023 (0.043)      0.082 (0.053)
[[lambda].sub.i]          --                 --

NSFSUP             0.222 (0.094) (b)  0.479 (0.085) (a)
[[lambda].sub.i]          --                 --

N                        126                126

Specification
                             (3)               (4)

Outcome                  Heckit ATE         Heckit TTE

TJEL                    -0.371 (1.84)      -4.07 (3.03)
[[lambda].sub.i]             --            2.63 (1.71)

TOP37                   0.807 (0.606)     -0.045 (1.01)
[[lambda].sub.i]             --           0.604 (0.566)

RESU                   -0.033 (0.167)     0.252 (0.277)
[[lambda].sub.i]             --           0.155 (0.156)

LARTS                  -0.037 (0.128)     -0.094 (0.212)
[[lambda].sub.i]             --           0.040 (0.119)

SLARTS                  0.002 (0.120)     0.009 (0.199)
[[lambda].sub.i]             --           -0.005 (0.113)

NBER                    0.088 (0.068)     0.110 (0.113)
[[lambda].sub.i]             --           -0.016 (0.064)

NSFSUP                0.417 (0.120) (a)   0.202 (0.198)
[[lambda].sub.i]                          0.152 (0.112)

N                            126               126

Control variables include dummy variables indicating
whether the treatment was received at Stanford, Temple,
Wisconsin, or Yale. The sample is truncated on the basis
of deleting control observations for which e(x) < 0.0319,
the lowest propensity score for the treated observations.

Standard errors are in parentheses.

N = number of observations.

(a) Significant at the 0.01 level.

(b) Significant at the 0.05 level.

(c) Significant at the 0.10 level.

Table 4. Propensity Score-Truncated
Estimates of Treatment Effects: e(x) < 0.3492

Specification            (1)               (2)

                   Propensity-Score  Propensity-Score
Outcome             Weighted ATE       Weighted TTE

TJEL               2.57 (1.25) (b)     1.95 (1.29)
[[lambda].sub.i]         --                 --

TOP37               1.01 (0.826)      0.900 (0.577)
[[lambda].sub.i]         --                 --

RESU               -0.164 (0.257)     -0.149 (0.231)
[[lambda].sub.i]         --                 --

LARTS              -0.010 (0.170)     0.055 (0.149)
[[lambda].sub.i]         --                 --

SLARTS             -0.035 (0.185)     0.030 (0.167)
[[lambda].sub.i]         --                 --

NBER                0.033 (0.090)     0.099 (0.095)
[[lambda].sub.i]         --                 --

NSFSUP              0.294 (0.226)    0.517 (0.161) (a)
[[lambda].sub.i]         --                 --

N                        39                 39

Specification
                            (3)                (4)

Outcome                  Heckit ATE        Heckit TTE

TJEL                    1.09 (2.93)       -2.55 (2.93)
[[lambda].sub.i]             --            1.87 (1.35)

TOP37                  0.548 (0.578)      -2.03 (1.25)
[[lambda].sub.i]             --          1.32 (0.579) (b)

RESU                   -0.061 (0.235)     0.388 (0.543)
[[lambda].sub.i]             --          -0.230 (0.251)

LARTS                  0.019 (0.160)     -0.116 (0.375)
[[lambda].sub.i]             --           0.069 (0.173)

SLARTS                 -0.024 (0.182)    -0.182 (0.426)
[[lambda].sub.i]             --          -0.081 (0.197)

NBER                   0.094 (0.100)      0.274 (0.232)
[[lambda].sub.i]             --          -0.093 (0.107)

NSFSUP                0.498 (0.169) (a)   0.259 (0.393)
[[lambda].sub.i]             --           0.122 (0.182)

N                            39                39

Control variables include dummy variables indicating
whether the treatment was received at Stanford, Temple,
Wisconsin, or Yale. The sample is truncated on the
basis of deleting those control observations for which
e(x) < 0.3492, the average propensity score for
the treated observations.

Standard errors are in parentheses.

N = number of observations.

(a) Significant at the 0.01 level.

(b) Significant at the 0.05 level.

Table 5. Propensity Score-Truncated Estimates of Treatment
Effects: [??](x) < 0.14

Specification            (1)                 (2)

                   Propensity-Score     Propensity-Score
Outcome              Weighted ATE         Weighted TTE

TJEL                  2.22 (1.01) (b)     1.72 (1.02) (c)
[[lambda].sub.i]          --                    --
TOP37                0.945 (0.627)       0.858 (0.482) (c)
[[lambda].sub.i]          --                    --
RESU                -0.161 (0.194)      -0.122 (0.185)
[[lambda].sub.i]          --                    --
LARTS               -0.059 (0.133)       0.019 (0.125)
[[lambda].sub.i]          --                    --
SLARTS              -0.041 (0.138)       0.022 (0.131)
[[lambda].sub.i]          --                    --
NBER                -0.006 (0.078)       0.078 (0.080)
[[lambda].sub.i]          --                    --
NSFSUP               0.250 (0.161)       0.494 (0.124) (a)
[[lambda].sub.i]          --                    --
N                         61                    61

Specification             (3)                  (4)

Outcome                Heckit ATE          Heckit TTE

TJEL                0.796 (1.21)         -2.22 (2.33)
[[lambda].sub.i]           --             1.79 (1.17)
TOP37               0.527 (0.587)        -1.27 (1.11)
[[lambda].sub.i]           --             1.06 (0.561) (c)
RESU               -0.110 (0.208)       -0.038 (0.406)
[[lambda].sub.i]           --           -0.042 (0.205)
LARTS              -0.034 (0.149)       -0.218 (0.292)
[[lambda].sub.i]           --            0.108 (0.147)
SLARTS             -0.015 (0.153)       -0.109 (0.301)
[[lambda].sub.i]           --            0.056 (0.152)
NBER                0.041 (0.103)        0.022 (0.201)
[[lambda].sub.i]           --            0.011 (0.101)
NSFSUP              0.440 (0.138) (a)    0.169 (0.268)
[[lambda].sub.i]           --            0.159 (0.135)
N                          61                 61

Control variables include dummy variables indicating whether the
treatment was received at Stanford, Temple, Wisconsin, or Yale.
The sample is truncated on the basis of deleting control
observations for which [??](x) < 0.14, the percent of economists in
the sample that actuall[y received the treatment (participated in
the AEASMP).

Standard errors are in parentheses.

N = number of observations.

(a) Significant at the 0.01 level.

(b) Significant at the 0.05 level.

(c) Significant at the 0.10 level.

Table 6. Heckit Estimates of Treatment on Treated: Logit Selection

Specification             (1)                  (2)

Outcome               Full Sample        [??](x) < 0.0319

TJEL                0.459 (4.61)         0.717 (2.93)
[[lambda].sub.i]    -2.82 (10.18)        -3.04 (6.36)
TOP37                1.39 (1.65)          1.92 (0.956) (b)
[[lambda].sub.i]    -2.46 (3.65)         -3.12 (2.07)
RESU                0.007 (0.262)        0.111 (0.266)
[[lambda].sub.i]   -0.050 (0.579)       -0.218 (0.578)
LARTS               0.001 (0.193)       -0.008 (0.203)
[[lambda].sub.i]   -0.101 (0.427)       -0.083 (0.441)
SLARTS             -0.032 (0.177)       -0.025 (0.191)
[[lambda].sub.i]    0.078 (0.392)        0.077 (0.415)
NBER                0.042 (0.096)        0.048 (0.108)
[[lambda].sub.i]    0.116 (0.213)        0.110 (0.234)
NSFSUP              0.691 (0.174) (a)    0.621 (0.189) (a)
[[lambda].sub.i]   -0.657 (0.385) (c)   -0.572 (0.412)
N                         180                  126

Specification             (3)                  (4)

Outcome             [??](x) < 0.3492      [??](x) < 0.14

TJEL                 2.55 (1.85)          2.31 (1.86)
[[lambda].sub.i]    -3.78 (3.47)         -3.93 (3.67)
TOP37                1.62 (0.803) (b)     1.64 (0.886) (c)
[[lambda].sub.i]    -2.79 (1.51) (c)     -2.91 (1.74) (c)
RESU               -0.027 (0.344)       -0.057 (0.322)
[[lambda].sub.i]   -0.089 (0.646)       -0.137 (0.634)
LARTS               0.037 (0.235)       -0.005 (0.232)
[[lambda].sub.i]   -0.046 (0.441)       -0.076 (0.457)
SLARTS             -0.056 (0.267)       -0.050 (0.238)
[[lambda].sub.i]    0.082 (0.502)        0.092 (0.469)
NBER                0.023 (0.146)       -0.023 (0.159)
[[lambda].sub.i]    0.183 (0.274)        0.165 (0.313)
NSFSUP              0.722 (0.241) (b)    0.670 (0.211) (a)
[[lambda].sub.i]   -0.583 (0.453)       -0.597 (0.416)
N                          39                 61

Control variables include dummy variables indicating whether
the treatment was received at Stanford, Temple, Wisconsin, or
Yale.

Standard errors are in parentheses.

N = number of observations.

(a) Significant at the 0.01 level.

(b) Significant at the 0.05 level.

(c) Significant at the 0.10 level.

The author would like to thank Charles Becker, Robert Feinberg, Philip Jefferson, Dan Newlon, and several anonymous referees for invaluable critical comments on earlier versions of this article. This article is based in part on confidential data obtained while the author was a Program Director for Economics at the National Science Foundation (NSF). All data are available upon request from the author. The views expressed are solely the author's and not necessarily those of the NSF.

Received April 2003; accepted December 2004.

(1) Traditionally, the AEASMP has not explicitly offered a curriculum that introduces and/or remediates the research capabilities of participants. However, since 1993, the program has applied for and received from the National Science Foundation, Research Experience for Undergraduates (REU) grants. As REU grants are designed to fund activities that enhance the research capabilities of undergraduates, the application and receipt of such funds by the AEASMP in recent years implies a recognition of how important research skills are for the successful completion by racial minorities in doctoral programs in economics.

(2) The underrepresentation of black economists on the faculties of colleges and universities is especially severe. Collins (2000), for example, reports that, as of 1995, Black American, Hispanics, and Native Americans were approximately 3% of economics doctorates employed.

(3) There is no formal published history of the AEASMP. The account provided here is based on reports and a National Science Foundation grant proposal authored by Charles Becker (2001a, 2001b, 2003)--the current director of the AEASMP at Duke University.

(4) WebCASPAR, located at http://caspar.nsf.gov, provides scientific and engnineering data on U.S. colleges and universities from the following sources: (1) NSF-NIH Survey of Graduate Students and Post doctorates in Science and Engineering, (2) NSF Survey of Research and Development Expenditures at Universities and Colleges, (3) NSF Survey of Federal Science and Engineering Support to Universities, Colleges, and Nonprofit Institutions, (4) NSF Survey of Federal Funds for Research and Development, (5) Survey of Earned Doctorates, (6) National Research Council U.S. Research-Doctorate Program ratings, and (7) NCES Integrated Postsecondary Education Data (IPEDS) surveys, and NCES Higher Education General Education System (HEGIS) surveys.

(5) Book reviews, such as those that appear in the Journal of Economic Literature, are not included. For each individual black economist, publication counts were measured as long as they were dated no later than December of 2000, so as to correspond with the known academic affiliation of the individual as of the 2000-2001 academic year.

(6) Although the EconLit database contains mostly economics journals, it does include journals such as Yale Law Review and the American Political Science Review, that are not economics journals proper. We do not distinguish between publications in economics journals and noneconomics journals--the only criterion is that the publication is listed in the EconLit database.

(7) Quality-adjusted publication weights are expensive to compute and existing measures severely reduce the numbers of journals that constitute the economics literature. The popular quality-weighting scheme of Laband and Piette (1994), for example, is based on only 130 journals in the EconLit database.

(8) We add the Journal of Economic Perspectives to the following list of top 36 economics journals considered by Scott and Mitias (1996): American Economic Review, Econometrica, Economic Inquiry, Economic Journal, Economica, Industrial and Labor Relations Review, International Economic Review, Journal of Business, Journal of Business and Economic Statistics, Journal of Development Economies, Journal of Econometrics, Journal of Economic Dynamics and Control, Journal of Economic History, Journal of Economic Theory, Journal of Finance, Journal of Financial Economics, Journal of Labor Economics, Journal of Human Resources, Journal of International Economics, Journal of International Money and Finance, Journal of Law and Economics, Journal of Law, Economics, and Organization, Journal of Legal Studies, Journal of Monetary Economics, Journal of Money, Credit, and Banking, Journal of Political Economy, Journal of Public Economics, Journal of Regional Science, Journal of Urban Economics, National Tax Journal, Public Choice, Quarterly Journal of Economics, Rand Journal of Economics, Review of Economic Studies, Review of Economics and Statistics, Southern Economic Journal.

(9) See the NBER at www.nber.org

(10) Nonparametric kernel density estimators of bootstrapped sampling distributions for treatment effects bounds--see Manski et al. (1992) for an example--require a choice of bandwidth. The optimal size of the bandwidth is inversely proportional to the inverse of the fifth root of the sample size (Kennedy 1998). Estimating the bandwidth proceeds by setting the bias and variance of the bandwidth equal. The resulting estimator is optimal in a mean-squared error sense, but the bias is equal to the variance (Rupert 1997). As the sample variance is inversely proportional to sample size, kernel density estimates based on small samples can have substantial bias.

(11) More generally, the unconfoundedness assumes that, after conditioning on observable pre treatment covariates, assignment into treatment can be viewed as random and unobservable characteristics do not determine assignment into treatment. This allows one to view treated and control observations in as if they are in a randomized experiment (Dehejia and Wahba 1998).

(12) More formally, Rosenbaum and Rubin (1983) define a propensity score as the coarsest function of the covariates that is a balancing score. The balancing score is a function of observed pretreatment covariates such that the conditional distribution of the covariates given the balancing score is the same for treated and control observations.

(13) Horowitz and Mammen (2002) show that estimating the necessary derivatives in nonparametric series estimators requires the use of kernel estimators. Our sample is insufficient to overcome the bias equal to variance restriction of mean-squared error optimal kernel density estimators.

(14) An increasingly popular alternative to the Heckit approach to estimating treatment effects when selection depends on both observed and unobserved characteristics is that of utilizing regression-discontinuity design (RDD), introduced by Thistlewaite and Campbell (1960). For example, Jacob and Lefgren (2004) have recently used this approach to identify the effects of remedial education on student achievement when selection into remedial education treatment is based on test scores. RDD identifies treatment effects by assuming that selection into a treatment is subject to some discontinuous threshold (e.g., test score cutoffs), where unobservable characteristics do not vary around some neighborhood of the discontinuity. In this context, the treatment is perfectly correlated with observable characteristics, allowing identification of the treatment effect. In the case of the AEASMP, implementing an RDD was not feasible, as there was no obvious feature of the process governing admission into the treatment that introduced discontinuities.

(15) BAMER is an imputed variable constructed on the basis of the full name of the black economist and the personal knowledge of the author. lf the name was not one with an implied Anglo-American origin (i.e., Curtis Williams or Sheila Hollins vs. Charles Okunade or Linda Amoateng), BAMER was imputed and coded as zero. HBCUBA was derived from various sources: faculty listings in college catalogs located at College Source Online (www.college.org), the American Economic Association Membership Directory, or college/university economics department webpages. YRPHD was derived from either information provided in the Prentice Hall Faculty Guide or from published dissertation abstracts online (www.firstsearch.org).

(16) We follow the method of Heckman, Tobias, and Vytlacil (2001) for estimating the average treatment effect (ATE), and the average gain from treatment for those selecting into the treatment--or the effect of treatment on the treated (TTE). In particular, for potential outcomes [Y.sup.1] = X [[beta.sup.i]] + [U.sup.1], [Y.sup.0] = X [[beta.sup.0]] + [U.sup.0], with election into treatment determined by a latent variable, [D.sup.*] = Z [theta] + [U.sup.D], the Heckit estimates in columns 3 and 4 are based on

ATE(x,z) = x([[beta].sup.1]-[[beta].sup.0])

TTE(x,z,D(z) = l) = x([[beta].sup.1]-[[beta].sup.0]) + ([[rho].sub.1] [[sigma].sub.1] - [[rho].sub.0] [[sigma].sub.0]) [phi](z[theta])/[PHI] (z[theta]),

where [rho].sub.i] = Corr([U.sup.i],[U.sup.D]), and [[sigma].sup.2.sub.i] = var([U.sup.i]) for i = 0, 1.

(17) The lowest propensity score among the treated units is 0.0469. One way to motivate this particular sample truncation is to view selection into the treatment as having some minimum threshold, which is characterized by the treated unit with the lowest propensity score. Thus, any control unit with a propensity score below the minimum among the treated units can be viewed as not being a comparable cohort, perhaps as a result of having dissimilar pretreatment covariates that determine selection into the treatment. This is a strategy that is often utilized in propensity-score matching estimators, as in Dehejia and Wahba (1998).

(18) The average propensity score for the treated observations is 0.2454. This views possible selection into treatment among the control observations as being governed by their match relative to the typical treated observation. A selection process like this would make sense if, over time, the program responsible for the treatment observes a correlation between certain characteristics and outcomes and desires to increase the efficacy of the treatment over time by increasing the standards for admission into the treatment.

(19) For the sample of 180 black Ph.D. economists in the sample, 14% actually participated in the AEASMP. Truncating control group observations with a propensity score less than 0.14 views selection into the AEASMP as an expected value based on the empirical frequency representing the minimum probability of participating in the entire population of black Ph.D. economists.

(20) The induced pipeline effects of black Ph.D. economist research productivity are far from trivial. For example, the analysis of Agesa, Granger, and Price (2000) suggests that, if economics faculty at historically black colleges and universities (HBCUs) averaged approximately two JEL-indexed articles over a 15-year period, which is close to the effect of treatment on the treated reported in the second column of Table 6 for publishing in top economics journals--an additional 129 black Ph.D. economists with baccalaureate degrees from HBCUs would have been created.

References

Agesa, Jacqueline, Maury Granger, and Gregory N. Price. 1998. Economics research at historically Black colleges and universities: Rankings and effects on the supply of black economists. Review of Black Political Economy 25:41-54.

Agesa, Jacqueline, Maury Granger, and Gregory N. Price. 2000. Economics research at teaching institutions: Are historically Black colleges and universities different? Southern Economic Journal 67:427-47.

Becker, Charles M. 2001a. Summer minority program. NSF proposal ses013528. Arlington, VA: National Science Foundation.

Becker, Charles M. 2001b. Summer program and minority scholarship program 1974-2001: History and alumni. University of Colorado at Denver.

Becker, Charles M. 2003. Annual report: American economic association minority scholarship program and summer training program. Department of Economics, Duke University, Durham, NC.

Burbridge, Lynn C., and Barbara A. P. Jones. 1989. 1988 directory of black economists. National Economic Association.

Collins, Susan M. 2000. Minority groups in the economics profession. Journal of Economic Perspectives 14:133-48.

Dehejia, Rajeev H., and Sadek Wahba. 1998. Causal effects in non-experimental studies: Re-evaluating the evaluation of training programs. NBER Working Paper 6586.

Feinberg, Robert M., and Gregory N. Price. 2004. The funding of economics research: Does social capital matter for success at the national science foundation? Review of Economics and Statistics 86:245-52.

Hasselback, James R. 2000. The Prentice Hall economics faculty guide 2000/2001. Englewood Cliffs, NJ: Prentice Hall.

Heckman, James. 1979. Sample selection bias as a specification error. Econometrica 42:153-62.

Heckman, James, Justin L. Tobias, and Edward Vytlacil. 2001. Four parameters of interest in the evaluation of social programs. Southern Economic Journal 68:211-23.

Hirano, Keisuke, Guido W. Imbens, and Gert Ridder. 2000. Efficient estimation of average treatment effects using the estimated propensity score. NBER Technical Working Paper 251.

Horowitz, Joel, and Enno Mammen. 2002. Nonparametric estimation of an additive model with a link function. Working paper cwp1902, the Institute for Fiscal Studies Department of Economics, University College of London.

Horvitz, Daniel G., and D. J. Thompson. 1952. A generalization of sampling without replacement from a finite universe. Journal of the American Statistical Association 47:663-85.

Imbens, Guido W. 2004. Nonparametric estimation of average treatment effects under exogeneity: A review. Review of Economics and Statistics 86:4-29.

Jacob, Brian A., and Lars Lefgren. 2004. Remedial education and student achievement: A regression-discontinuity analysis. Review of Economics and Statistics 86:226-44.

Kennedy, Peter. 1998. A guide to econometrics. 4th edition. Cambridge, MA: MIT Press.

Laband, David N., and Michael J. Piette. 1994. The relative impact of economics journals: 1970-1990. Journal of Economic Literature 32:640-66.

Lalonde, Robert. 1986. Evaluating the econometric evaluations of training programs. American Economic Review 76:604-20.

Lechner, Michael. 1999. Nonparametric bounds on employment and income effects of continuous vocational training in east germany. Econometrics Journal 2:1-28.

Leeds, Michael A. 1992. Who benefits from affirmative action? The case of the AEA summer minority program 1986-1990. Journal of Economic Perspectives 6:149-56.

Manski, Charles F. 1990. Nonparametric bounds on treatment effects. American Economic Review 80:319-23.

Manski, Chares F., Gary D. Sandefur, Sara McLanahan, and Daniel Powers. 1992. Alternative estimates of the effect of family structure during adolescence on high school graduation. Journal of the American Statistical Association 87:25-37.

Newey, Whitney K. 1994. The asymptotic variance of semiparametric estimators. Econometrica 62:1349-82.

Rosenbaum, Paul R., and Donald B. Rubin. 1983. The central role of the propensity score in observational studies for causal effects. Biometrika 70:41-55.

Rosenbaum, Paul R., and Donald B. Rubin. 1984. Reducing bias in observational studies using subclassification on the propensity score. Journal of the American Statistical Association 79:516-24.

Rubin, Donald B. 1974. Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of Educational Psychology 66:688-701.

Ruppert, David. 1997. Empirical-bias bandwidths for local polynomial nonparametric regression and density estimation. Journal of the American Statistical Association 92:1049-62.

Schmidt, Christoph M., and Jochen Kluve. 2001. Comparing the comparable Nobel Prize Winner James J. Heckman. Journal of Population Economics 14:1-5.

Scott, Loren C., and Peter M. Mitias. 1996. Trends in rankings of economics departments in the U.S.: An update. Economic Inquiry 34:378-400.

Thistlethwaite, Donald, and Donald Campbell. 1960. Regression-discontinuity analysis: An alternative to the ex post facto experiment. Journal of Educational Psychology 51:309-17.

Wachtel, Howard L. 2000. How the national science foundation funds research in economics. Challenge 43:20-30.

Gregory N. Price, Mississippi Urban Research Center, Jackson State University, P.O. Box 17309, Jackson, MS 39217, USA; E-mail: [e-mail].