Printer Friendly

A call for a truce in the DGU war.

For almost a decade scholars have been debating about how

many defensive gun uses (DGUs) occur annually. Gary Kleck and

colleagues,(1) citing a series of polls culminating in the 1993 Kleck-Gertz

survey, argue that at least 2.55 million people use a firearm for

protection against criminals each year. Hemenway and others,(2) relying on

the National Crime Victimization Surveys (NCVSs), contend that only

about 55,000 to 80,000 victims use guns against offenders in a given

year. The estimates are wide apart and their academic champions

staunchly defend their respective figures as correct and accurate,

while dismissing the opposing figures as invalid and implausible.

Neither side seems to be willing to give ground or see their

opponents' point of view. This is unfortunate since there is good reason to

believe that both sides are off-the-mark. Below the main shortcomings

of the two approaches and some of the keys issues of contention are


First, it appears that the estimates of the NCVSs are too low.

There are two chief reasons for this. First, only DGUs that are

reported as part of a victim's response to a specified crime are

potentially covered. While most major felonies are covered by the NCVSs, a

number of crimes such as trespassing, vandalism, and malicious

mischief are not. DGUs in response to these and other events beyond the

scope of the NCVSs are missed.

Second, the NCVSs do not directly inquire about DGUs. After a

covered crime has been reported, the victim is asked if he or she "did

or tried to do [anything] about the incident while it was going on.

Indirect questions that rely on a respondent volunteering a specific

element as part of a broad and unfocused inquiry uniformly lead to

undercounts of the particular of interest.(3)

However, some other proposed reasons for under-reporting on

the NCVSs are questionable. The claim that DGUs are

under-reported because the NCVSs suffer from "the taint of being conducted

by, and on behalf of employees of the federal government"(4) and that

respondents see themselves in effect as "speaking to a law

enforcement arm of the federal government"(5) is improbable. The survey

literature does not indicate that Bureau of the Census surveys are held

in special suspicion.(6) If anything, it indicates that cooperation is

greater than usual in part because of the high quality of Census

interviewers and because most people accord the Bureau of the Census

more legitimacy than given to other surveys.(7)

Second, the estimates of the Kleck-Gertz study and other

cross-sectional surveys using a direct question are too high. First, unlike the

panel NCVs which uses bounded recall to minimize telescoping (i.e.,

the misreporting of past events as having occurred within a more

recent specified time period), nothing in these surveys mitigates against

such over-reporting. Kleck and Gertz (K-G) are correct to note that

forgetting would tend to off-set errors from telescoping,(8) but these

two cognitive errors are rarely balanced. While no definitive study of

the relative telescoping versus forgetting rate for DGUs exists, given a

one-year reference period and the saliency of DGUs, it is likely that

telescoping is greater than forgetting.(9)

Second, there is a significant amount of sampling error around

the direct DGU estimates. While K-G are correct that, broadly

speaking, all of the direct estimates are compatible with one another (i.e.,

probably mostly within the confidence intervals),(10) this is in part

because the sampling variation is often rather large. To say that the

high-end K-G estimate is not statistically implausible, given results

from other similar surveys, is not to say that it is correct. What is

needed is a meta-analysis that takes comparable estimates from similar

surveys and produces the best overall estimate. Since the K-G estimate

is near the high end of the range of estimates based on national

samples covering specific reference periods, this means that the best

estimate is lower. Giving equal weight to all estimates,(11) the composite

annual estimate based on one-year recall would be 1.81-2.01 million,

the annual estimate based on five-year recall period would be about

1.34-1.38, using the K-G multiple occurrence adjustment, and around

0.9-1.0 without that revision (Table 1).(12)


Number of Adults with DGUs per annum


 Based on One Based on Five

 Study Variant Year Recall Year Recall

K-G 1993(13) A 2.55 1.88

K-G 1993 B 2.16 1.68

K-G (Hart, 1981)(14) 1.80

K-G (Mauser, 1990) 1.49

K-G (Tarrance, 1994) 0.76

NSPOF(15) 1 1.46 0.65

NSPOF(16) 2 1.46 0.97

Third, as Hemenway (H) points out, the K-G estimate is likely to

suffer from false positives,(17) although the situation is not nearly as

clear as H asserts. As K-G note in their response to H's critique, the

basic medical misreport model assumes that the errors are random.(18)

As such, the rarer the event the greater the over-reports because there

are many more true negatives that can be "accidently" misclassified as

false positives than there are true positives that could by chance be

misreported as false negatives. In medicine, this problem is addressed

by a definitive and independent follow-up test to confirm or refute the

more error-prone, screening test. K-G in effect argue that they apply

such a test, by asking up to nineteen follow-up questions to verify that

the reported positive is a true rather than a false positive.(19) To their

credit, they use these follow-ups to eliminate both probable and some

possible false positives.

But there are two serious limitations to this procedure. First, the

confirming test is not an independent test. Just the opposite. There

is likely to be correlated error. If a person misreports a DGU on the

screener, he or she is likely to misreport it in the follow-ups. For

example, if respondents misreport real DGUs as occurring within the

referenced time period due to telescoping, then in the follow-up

questions they would report the details of the misreported incidents.

Second, neither the screener nor the follow-up questions are

non-reactive. While people cannot consciously affect typical medical

tests, they can consciously or unconsciously distort survey responses.

For example, if respondents intentionally misreport a DGU, they

could merely continue the deception in the follow-up items.

The follow-up questions are essential for collecting useful

information of the characteristics of DGUs and can help to weed out some

misreports. But they will only reveal a limited number of errors such

as if the interviewer misheard or miscoded the initial response as a

"Yes" or when the respondent misunderstood the screening question

and reversed the initial response in light of the follow-up questions

that made clear the meaning of the initial question. Many other

misreports will be undetected.

The key aspect of the argument over how much distortion occurs

and in what direction it leans concerns respondents' motivation to

report accurately. H contends that there is a strong social desirability

effect that would lead people to over-report DGUs (either via

exaggeration, increased telescoping, or outright fabrication).(20) He believes

people see DGUs as heroic acts that reflect well on themselves.(21) K-G

however in their rebuttal contend that the social desirability effect

works in the reverse.(22) They repeatedly argue: (1) that "most of the

reported DGUs involved illegal behavior on the part of Rs."(23) and

"DGUs typically involve criminal behavior"(24); and (2) as a result, that

"people are far more likely to fail to report illegal behavior in which

they have engaged than they are to falsely report illegal behaviors in

which they have not engaged."(25) Both authors are correct about the

impact that social desirability would have if respondents on balance

saw DGUs as respectively heroic or criminal.

Unfortunately, there is no empirical evidence as to how people

view DGUs. To establish social desirability effects two types of

evidence are needed: (1) validation studies that document that people

consistently over or underreport the true level of a behavior; and (2)

studies that indicate that self-presentation (rather than cognitive error

such as misunderstanding the question or some imbalance of

forgetting and telescoping) is the source of the misreports. No such studies

apparently exist for DGUs.

However, one part of K-G's argument about possible social

desirability effects is questionable.(26) Neither their study (at least as

reported), nor any other study really demonstrates that "most" DGUs

involve illegal actions or "typically" or "usually" involve criminal

behavior on the victim's part. K-G's data are too imprecise to ascertain this

in general.(27)

Finally, as K-G acknowledge, various statistics from their survey

are wrong, questionable, or severely limited: (1) DGUs by household

members other than R are under-reported;(28) (2) DGUs estimates

based on five-year recall are inconsistent with and lower than the

one-year estimates;(29) (3) the wounding rate is too high;(30) (4) the incident

hit rate is too high;(31) and (5) in general, figures based on just the

DGU cases are subject to considerable sampling variation and the less

accurate five-year reports and household incidents have to be relied

upon in order to get enough observations for any analysis of the

characteristics of DGUs.(32)

Another questionable finding, and one echoed by the NSPOF,(33)

is that a high proportion of DGUs are carried out by women. K-G find

that 46.3% of defenders are women(34) and the NSPOF finds that

41.2% are women.(35) Given that the best estimate is that only 20-21%

of gun owners are women,(36) this means that women are twice as likely

to use a gun defensively as one would expect. When one looks at the

small number of DGUs that results in justifiable homicides, only

13.8% are committed by women.(37) If women are 21% of gun owners

and approximately 14% of those who lawfully kill someone with a gun,

it seems improbable that they would make up 41-46% of all DGUs.

In addition, further concerns come from figures relating to the

level of gun usage in crimes, the absence of guns in some DGU

households, and how many lives are thought to have been saved by DGUs.

Likewise, in the similar NSPOF internal confusion and contradictions

are noted.(38)

K-G are right to note that the details of DGUs are based on only

222 cases (or even less when sub-group analysis is performed) and that

these estimates are much more subject to sampling variation than are

the estimates of the level of DGUs based on 4,977 cases.(39) But the

large number of doubtful figures about the particulars of DGUs do

raise concerns about the reliability and accuracy of such reports. If

many of the details about DGUs are suspect, this suggests that recall is

unreliable and that the counts themselves may also be inaccurate.

However, various other arguments against the K-G estimates do

not shed much light on the situation. Figures on UFOs and the

mentally ill are not particularly relevant or useful.

If we factor in some of the probable over- and underestimates

affecting the NCVS and K-G 1993 survey, the widely divergent figures

on DGUs draw much closer together. The latest figures from the

NCVS indicate 108,000 DGUs per annum.(40) If this is adjusted for a

50% under-reporting due to not directly asking for DGUs, this

increases the estimate to 216,000. Next, research by Cook and Ludwig

suggests that perhaps 16-42% of DGUs involve crimes not covered by

the NCVS.(41) Adding in these would raise DGUs to 256,500-373,000.

Similarly, using the average of the K-G one-year lower (B)

estimate and the NSPOF figure gives a starting estimate of 1,810,000.

Assuming a net cognitive over-reporting (telescoping - forgetting) of

50%,(42) reduces the figure to 1,210,000.(43) These estimates should

draw even closer together if other measurement errors could be

factored in.(44) But even as they stand, the gulf has been narrowed from

30+:1 to 3.2-5.6:1.

Of course, the above calculations are based on reasonable, but

mostly unconfirmed, estimates of various error parameters. What is

needed is less argumentation and speculation and more and better

data. First, some additional information can be gained by refined

analysis of the existing surveys (K-G 1993, NSPOF, NCVS, etc.). Each

of the surveys should be fully documented and archived at the Roper

Center, University of Connecticut, and the Interuniversity Consortium

for Political and Social Research, University of Michigan, for any

researcher to use.(45)

Second, more studies are needed. These should include: (1)

validation studies specifically designed to ascertain whether a social

desirability bias exists for DGUs; (2) studies that experimentally vary factors

that are believed to inflate or deflate DGUs reports to see (a) how

robust reports are and (b) whether they are affected by the

hypothesized factors; (3) taped descriptions of reported DGUs with detailed

probes so that one can determine exactly what transpired, including

such issues as whether (a) a criminal threat existed, its nature, and

seriousness, (b) the DGU was probably legal, and (c) accounts are

accurate and truthful; (4) trying alternative methods for measuring

DGUs that might lessen both any self-presentation bias and cognitive

error. One possibility would be to ask people whether they had

handled or fired a gun in the last year and then ask about for what

purpose it was used (e.g., hunting, target shooting, self-defense, etc.); and

(5) the use of refined, direct experience questions on a large, high

quality, panel survey with explicit corrections for telescoping. Adding

a few questions to the NCVS would be the easiest way to achieve this.

Only by such further careful, empirical research will the errors in

measuring DGUs be understood and the true level of DGUs


(1) Gary Kleck & Marc Gertz, The Illegitimacy of One-Sided Speculation: Getting

the Defensive Gun Use Estimate Down, 87 J. Crim. L. & Criminology 1446 (1997);

Gary Kleck & Marc Gertz, Armed Resistance to Crime: The Prevalence and Nature

of Self-defense with a Gun, 86 J. Crim. L. & Criminology 150 (1995).

(2) David Hemenway, Survey Research and Self Defense Gun Use: An Explanation of

Extreme Overestimates, 87 J. Crim. L. & Criminology 1430 (1997).

(3) Seymour Sudman & Norman Bradburn, Asking Questions: A Practical Guide to

Questionnaire Design, 36-45 (1982); Tom W. Smith, Trends in Voluntary Group

Membership, 34 Am. J. Pol. Sci. 646, 647-52 (1990).

(4) Kleck & Gertz, Armed Resistance to Crime, supra note 1, at 166.

(5) Id. at 155.

(6) See, e.g., Richard Kulka et al., Self-Reports of Time Pressures, Concerns

for Privacy, and Participation in the 1990 Mail Census, Paper presented to the

Annual Research Conference, U.S. Bureau of the Census, Arlington, Virginia

(1991); Panel on Privacy and Confidentiality as Factors in Survey Response,

Privacy and Confidentiality as Factors in Survey Response (1979).

(7) See Kulka, supra note 6; Privacy and Confidentiality as Factors in Survey

Response, supra note 6.

(8) Kleck & Gertz, Armed Resistance to Crime, supra note 1, at 163.

(9) See generally Ronald Andersen et. al., Total Survey Error: Applications to

Improve Health Surveys (1979); William Foddy, Constructing Questions for

Interviews and Questionnaires (1993); J. Neter & J. Waksberg, A Study of

Response Errors in Expenditures Data from Household Interviews, 59 J. Amer.

Stat. Ass'n 18 (1964).

(10) Kleck & Gertz, Armed Resistance to Crime, supra note 1, at 166.

(11) A more statistically sophisticated meta-analysis could use varying weights

for the studies to adjust for sample size and other factors.

(12) K-G take the five-year rate, divide it by 5 and increase this result by

about 50% to account for multiple occurrences during the five years which might

have occurred in different years. See Kleck & Gertz, Armed Resistance to Crime,

supra note 1, at 166.

(13) "K-G 1993" is K-G's own study and estimates. See id. at tbl.2 (A & B

estimates as defined therein).

(14) "K-G" (Hart, Mauser, and Tarrance) are K-G's 1993 estimates from other

studies. See id. at tbl.1.

(15) Philip J. Cook & Jens Ludwig, The Private American Arsenal: Results of a

Comprehensive National Survey on Firearms Ownership and Use (1996) (unpublished

report on file with author). Cook and Ludwig analyzed the National Survey of

Private Ownership of Firearms (NSPOF). Estimate 1 takes their five year

estimate and divides by five to get an annual estimate.

(16) Id. Estimate 2 takes estimate 1 and increases by 50% to account for

possible multiple DGUs that might have occurred in different years. This is

approximately the adjustment that K-G used to get their annual estimates from

five-year reports. See Kleck & Gertz, Armed Resistance to Crime, supra note 1,

at 166.

(17) Hemenway, supra note 2, at 1435-37.

(18) Kleck & Gertz, 7he Illegitimacy of One-Sided Speculation, supra note 1, at


(19) Id. at 1449-50.

(20) See Hemenway, supra note 2, at 1438-40.

(21) Id. at 1438.

(22) The issue is raised in a much more limited fashion in the initial article.

See Kleck & Gertz, Armed Resistance to Crime, supra note 1, at 156, 172-74. The

authors do not state it so strongly, do not claim that DGUs are in general

illegal acts, and focus on the impact of willingness to report on the NCVS, not

on their own survey.

(23) Kleck & Gertz, The Illegitimacy of One-Sided Speculation, supra note 1, at


(24) Id. at 1452.

(25) Id. at 1459.

(26) See Kleck & Gertz, Armed Resistance to Crime, supra note 1, at 163.

(27) K-G state: "We made no efforts to assess either the lawfulness or morality

of the R's defensive actions." Id. But elsewhere K-G infer that some of the

used guns are illegal or the victim is not legally entitled to use or possess

the weapons, that other guns were illegally carried prior to use, and that in

other cases the victim was actually the aggressor. See id. at 156. There is no

real evidence on the first and last points and the evidence on the middle point

is inconclusive, but probably points to most DGUs not involving illegal

carrying. K-G's estimate that 36-64% involved illegal gun carrying is on the

high side. See id. at 174.

(28) Id. at 165.

(29) Id.

(30) Id. at 173.

(31) Id.

(32) Id. at 172-73.

(33) Cook & Ludwig, supra note 15, at 104.

(34) Kleck & Gertz, Armed Resistance to Crime, supra note 1, at tbl.4.

(35) Cook & Ludwig, supra note 15, at 104.

(36) Tom W. Smith & Robert J. Smith, Changes in Firearm Ownership Among Women,

1980-1994, 86 J. Crim L. & Criminology 133, 143 (1995).

(37) Based on the analysis of the 1976-1990 Supplementary Homicide Reports of

the Federal Bureau of Investigation (FBI) reported in Abraham N. Tennenbaum,

Justifiable, Homicides by Civilians in the U.S., 1976-1990: An Fxploratory

Analysis, tbl.11 (1993) (unpublished Ph.D. dissertation, University of

Maryland) (on file with author). Unpublished figures from special data run by

the FBI for 1991-1995 for the author show similar results.

(38) Cook & Ludwig, supra note 15, at 91.

(39) Kleck & Gertz, Armed Resistance to Crime, supra note 1. at 162.

(40) Philip J. Cook et al., The Gun Debate's New Mythical Number: How Many

Defensive Uses Per Year?, 16 J. Pol'y Analysis & Mgmt. 463, 468 (1997).

(41) Cook & Ludwig, supra note 15. Based on a very small sample, they found

that 16% of DGUs involved trespassing and 47% involved no threat, attack, or

injury. Id. at 109. K-G seem to indicate that fewer DGUs involve non-NCVS

crimes (3.7% only trespassing and 9.5% "other crimes"). Kleck & Gertz, Armed

Resistance to Crime, supra note 1, at 185-86. They also state that "a large

share of the incidents covered by our survey are probably outside the scope of

incidents that realistically are likely to be reported to the NCVS ...." Id. at


(42) K-G suggest that telescoping might lead to an overestimate of 21%. Kleck &

Gertz, Armed Resistance to Crime, supra note 1, at 171. While no definitive

figure exists, work on the NCVS and other surveys suggest the rate is more

likely to be around 50%. See Ronald Andersen et al., supra note 9, at 19-20,

32-37, 42-50, 52-74; David Cantor, Substantive Implications of Longitudinal

Design Features: The National Crime Survey As a Case Study, in Panel Surveys

(Daniel Kaspryz et al. eds. 1989); Neter & Waksberg, supra note 9.

(43) Or to an even lower figure if annual estimates based on the five-year

recall questions are used.

(44) For example, K-G acknowledge that up to 10% of DGUs might really represent

instances of "mutual combat." Kleck & Gertz, The Illegitimacy of One-Sided

Speculation, supra note 1, at 1450. Also, if a positive social desirability

bias exists (as H argues) the K-G number would come down much further. See

Hemenway, supra note 2, at 1430-31. But if K-G are correct about there being a

negative social desirability bias, then the above adjusted estimates would

already be too low and both their and the NCVS estimates would rise.

(45) The NCVS is in the public domain already.
COPYRIGHT 1997 Northwestern University, School of Law
No portion of this article can be reproduced without the express written permission from the copyright holder.
Copyright 1997 Gale, Cengage Learning. All rights reserved.

Article Details
Printer friendly Cite/link Email Feedback
Title Annotation:defensive gun use
Author:Smith, Tom W.
Publication:Journal of Criminal Law and Criminology
Date:Jun 22, 1997
Previous Article:The illegitimacy of one-sided speculation: getting the defensive gun use estimate down.
Next Article:Copyrights, criminal sanctions and economic rents: applying the rent seeking model to the criminal law formulation process.

Terms of use | Privacy policy | Copyright © 2018 Farlex, Inc. | Feedback | For webmasters