# Comment on James J. Heckman, "Instrumental Variables: A Study of Implicit Behavioral Assumptions Used in Making Program Evaluations".

Guido W. Imbens [*]ABSTRACT

In a recent paper in this journal, Heckman discussed the use of instrumental variables methods in evaluation research and our local average treatment effects (LATE) interpretation of instrumental variables estimates. This comment provides additional background for Heckman's paper, and a review of our rationale for focusing on LATE. We also show that a set of assumptions proposed by Heckman as an alternative to the LATE assumptions are not compatible with either latent-index assignment models or the definition we proposed for an instrument.

I. Background

Heckman (1997) discusses the merits of applying instrumental variables methods to evaluation problems and the "Local Average Treatment Effect" (LATE) interpretation of the IV estimand developed by Imbens and Angrist (1994). Our first purpose in writing this comment is to draw readers' attention to our earlier paper (Angrist, Imbens, and Rubin 1996a, AIR from hereon) which is followed by published comments from Heckman (1996) and others and our rejoinder (Angrist, Imbens, and Rubin 1996b). Sections III, IV, and V Heckman's JHR paper cover much the same ground as his comment on AIR. Given this overlap, a surprising feature of the JHR piece is that it makes no reference to our earlier exchange.

Heckman's exploration of how LATE can be related to economic models of program participation in Section VI of the JHR paper goes beyond our earlier exchange and we find this new material especially useful and interesting. At the same time, we would like to point out that the interpretation of instrumental variables estimates in models with multi-valued treatments like schooling, a question Heckman explores in Section VII, was originally discussed in Angrist and Imbens (1995). Card (1995) also interprets instrumental variables estimations of returns to schooling in a structural model very similar to Heckman's. Another paper (Angrist, Imbens, and Graddy 1995) discusses a related problem for continuous treatments. Other relevant work is the criticism of instrumental variables assumptions from a viewpoint of utility maximizing agents in Vella and Verbeek (1995). The concept of the effect of treatment on the treated, a cornerstone of the JHR piece, dates back at least to Peters (1941), Belson (1956) and Rubin (19 73).

We also note that Heckman's JHR piece appears to mark a break from his earlier support of IV methods. See, for example, Heckman (1990), where he writes "there is accumulating evidence that instrumental variables procedures "work" [p. 317], and Heckman and Robb [1985] "The instrumental variables estimator is the least demanding in the a priori conditions that must be satisfied for its use" [p. 185]. We agree with some aspects of Heckman's new, more cautious attitude, but his characterization of our views is exaggerated. In particular Angrist's (1990) discussion does not make a claim for "ideal instrumental variables" (Heckman 1997, p.5). In fact, Angrist (1990, Section V, "Caveats") included a discussion of possible violations of the identifying assumptions underlying use of the lottery as an instrument for veteran status. AIR explores some of these issues further and explains why we think the lottery is nevertheless a plausible instrument once the LATE interpretation is spelled out.

II. Scientific Issues

Our focus on LATE is not motivated by the view that it is the only average causal effect of interest. Rather, we view it as the only effect that can be estimated credibly and consistently in an instrumental variables setting. Similarly, the population subjected to treatment in randomized trials is not always the only population of interest. For example, some trials are conducted only on men even though the population of interest includes both men and women. It seems appropriate in this case to report the results as applying to men and leave the question of whether they apply to women open, to be resolved, perhaps, by further evidence or theoretical reasoning. [1] Likewise, under the AIR assumptions, one can estimate the average treatment effect for compliers. For the two groups of noncompliers, never-takers and always-takers, one cannot estimate the average treatment effect. Never-takers, like the millionaires in Heckman's training program, are people who, at least within the context of the study, are never observed receiving the treatment. The LATE philosophy is to report estimates of the average treatment effect for compliers, with a clear statement that it is an average effect for the subpopulation of compliers. Our work represents an attempt to avoid confusing the assumptions necessary for estimation of the average effect for compliers with the assumptions necessary for extrapolation to noncompliers.

A second issue involves the assumption suggested by Heckman as an alternative to our exclusion restriction. Using Heckman's notation and setup, simplified by omitting regressors, we have:

(1) [Y.sub.0] = [[micro].sub.0] + [U.sub.0],

(2) [Y.sub.1] = [[micro].sub.1] + [U.sub.1]

where [U.sub.0] and [U.sub.1] are deviations from the population mean of [Y.sub.0] and [Y.sub.1] respectively. An identifying assumption Heckman has suggested here and earlier is Assumption (C-1-b) (Assumptions A1 and A2 in Heckman 1996). [2]

(3) E[[U.sub.0] + D([U.sub.1] - [U.sub.0] - E[[U.sub.1] - [U.sub.0]/D = 1])/Z] = 0.

If the treatment effect is constant across units this assumption reduces to assuming mean independence of [U.sub.0] and the instrument. With heterogenous treatment effects, however, this assumption is very different from the exclusion restriction in AIR which requires that [Y.sub.0] and [Y.sub.1] be jointly independent of Z. The following two results demonstrate that Heckman' s assumption is neither necessary nor sufficient for instrumental variables methods to be applicable. Not only is Equation 3 a very strong assumption, it is false by construction in most latent index models:

Result 1 (Necessity)

In an econometric selection model with Equations 1 and 2 for the outcome equations, and a single linear index model for the selection equation:

(4) D = 1 {[[gamma].sub.0] + [[gamma].sub.1] . Z + [eta] [greater than] 0},

where Z is independent of [U.sub.0] and [U.sub.1], and [eta] is a continuously distributed latent error term independent of the instrument, Heckman's Assumption 3 can only be satisfied if [eta] is independent of [U.sub.1] - [U.sub.0].

This would be true for example in the random selection case ([eta] independent of [U.sub.0] and [U.sub.1], but that is of little interest in an instrumental variables setting because simple treatment-control mean differences are then consistent for the average treatment effect. Heckman's Assumption 3 is unnecessary, however, because standard IV methods still estimate the local average treatment effect in this case:

(5) E[[Y.sub.1] - [Y.sub.0]/[[gamma].sub.0] + [eta] [less than] 0 [less than] [[gamma].sub.0] + [[gamma].sub.1] + [eta]],

In fact, Heckman recognizes this in footnote 11. [3]

Finally, in contrast with our approach, which casts identifying assumptions in terms of the independence between instruments and counterfactuals, Heckman's assumption is not sufficient for Z to be an instrument according to the definition we used in AIR.

Result 2 (Sufficiency)

Suppose [U.sub.0] is independent of both D and the binary instrument Z. Suppose in addition that [U.sub.1] = [alpha] and Pr(D = 1Z) = p . Z. Then Heckman's Assumption 3 is satisfied.

In this case, even though Heckman's assumption is satisfied and the average effect on the treated (SATE) is identified, it seems unattractive to refer to Z as an instrument since it has a direct effect on the outcome of interest (that is, other than through D).

We hope this comment clarifies our view of the role of instrumental variables methods can play in evaluation research.

Joshua D. Angrist is a professor of economics at the Massachusetts Institute of Technology and a research assistant at the National Bureau of Economic Research. Guido W. Imbens is a professor of Economics at the University of California, Los Angeles, and a research associate at the National Bureau of Economic Research.

(1.) For example, Angrist (1990) outlines an economic model explaining why military service reduces the earnings of veterans. Angrist and Krueger (1994) argue that the same model applies to World War II veterans.

(2.) Again, simplified here by omitting regressors.

(3.) In the draft lottery case, for example, this is the effect of military service on those who served because they were at risk from being drafted. This parameter is relevant for evaluating the effects of compulsory military service on veterans.

References

Angrist, J. D. 1990. "Lifetime Earnings and the Vietnam Era Draft Lottery: Evidence from Social Security Administrative Records." American Economic Review 80:313-35,

Angrist, J. D., K. Graddy, and G. W. Imbens. 1995. "Non-parametric Demand Analysis with an Application to the Demand for Fish." National Bureau of Economic Research, Technical Working Paper 178.

Angrist, J., and G. Imbens. 1995. "Two-Stage Least Squares Estimation of Average Causal Effects in Models with Variable Treatment Intensity." Journal of the American Statistical Association 90(430):43 1-42.

Angrist, J., G. Imbens and D. Rubin. 1996a. "Identification of Causal Effects Using Instrumental Variables." Journal of the American Statistical Association 91(434):4.44-72.

Angrist, J., G. Imbens, and D. Rubin. 1996b. Reply to Discussants of "Identification of Causal Effects Using Instrumental Variables." Journal of the American Statistical Association 91(434):468-72.

Angrist, J. D., and A. Krueger. 1994. "Why do World War II Veterans Earn more than Nonveterans." Journal of Labor Economics 12:74-97.

Belson, W. A. 1956. "A Technique for Studying the Effects of a Television Broadcast." Applied Statistics 5:195-202.

Card, D. 1995. "Earnings, Schooling and Ability Revisited." In Research In Labor Economics, Vol 14, ed. Solomon and Polachek, 23-48. Greenwich, Conn.: AI Press.

Heckman, J. 1990. "Varieties of Selection Bias." American Economic Review 80(2):313-18.

_____. 1996. Comment on: Angrist, Imbens, and Rubin, "Identification of Causal Effects Using Instrumental Variables." Journal of the American Statistical Association 91(434):459-62.

_____. 1997. "Instrumental Variables: A Study of Implicit Behavioral Assumption Used in Making Program Evaluations." Journal of Human Resources 32(2):441-62.

Heckman, J., and R. Robb. 1985. "Alternative Methods for Evaluating the Impact of Interventions." In Longitudinal Analysis of Labor Market Data, ed., J. Heckman and B. Singer Cambridge: Cambridge University Press.

Imbens, G. W., and J. D. Angrist. 1994. "Identification and Estimation of Local Average Treatment Effects." Econometrica 62(2):467-76.

Peters, C. C. 1941. "A Method of Matching Groups for Experiments With No Loss of Population." Journal of Educational Research 34:606-12.

Rubin, D. B. 1973. "Matching to Remove Bias in Observational Studies." Biometrics 29:159-83.

Vella, F., and M. Verbeek. 1995. "Estimating and Interpreting Models with Endogenous Treatment Effects." Journal of Business and Economic Statistics. Forthcoming.

Printer friendly Cite/link Email Feedback | |

Author: | Angrist, Joshua D.; Imbens, Guido W. |
---|---|

Publication: | Journal of Human Resources |

Article Type: | Statistical Data Included |

Date: | Sep 22, 1999 |

Words: | 1769 |

Previous Article: | The Effect of Child Care Characteristics on Child Development. |

Next Article: | Instrumental Variables. |

Topics: |