August 23, 2012

Lies of the Gun Lobby (part 2)

Lie #2: Concealed-carry laws reduce crime.

From the NRA:

Studying crime trends in every county in the U.S., economist John Lott and David Mustard concluded, “allowing citizens to carry concealed weapons deters violent crimes. . . . [W]hen state concealed handgun laws went into effect in a county, murders fell by 8.5 percent, and rapes and aggravated assaults fell by 5 and 7 percent.”[1]

Lott and Mustard derived their numbers using econometrics, specifically, a multiple regression model.[2] A multiple regression model is a statistical tool that, in theory at least, helps researchers determine the role that different factors play in causing an event. The reason for using such a model is obvious enough, as most events can have any number of causes. So Lott and Mustard employed a regression model in an attempt to determine the extent to which crime rates from were effected by concealed-carry laws and the extent to which they were effected by other factors. 

But there are serious problems with econometrics. As Ted Goertzel has noted, multiple regression would work “[i]f one had perfect measures of all the causal variables.” However, the fact is that “the data are never good enough. Repeated efforts to use multiple regression to achieve definitive answers to public policy questions have failed.” Goertzel notes that shortly after Lott and Mustard published their study, economists Dan Black and Daniel Nagin “published a study showing that if they changed the statistical model a little bit, or applied it to different segments of the data, Lott and Mustard’s findings disappeared.”[3]

In 2003 Yale economists Ian Ayers and John Donohue made a regression model containing seven additional years of data.[4] Their model determined that concealed-carry states actually had “higher crime rates in eight of the nine crime categories.” When they added “controls for other factors that might be including crime over this period,” they found that concealed-carry states had higher crime rates in seven of the nine categories.[5], [6]

Ayers and Donohue found that their thesis was amplified when they expressed their findings on a state-specific basis. (Lott and Mustard had expressed their findings in aggregate terms.)[7] When doing this, they found that concealed-carry laws “increased crime in substantially more jurisdictions than they decreased crime.” Breaking it down further, they found that there were “almost twice as many jurisdictions with an estimated increase in violent crime (fifteen) as those with an estimated decrease (nine)” and that there were “eight states with a statistically significant increase in murder while only four states exhibit[ed] a statistically significant decrease.” 

None of this, I should point out, necessarily means that concealed-carry laws increase crime. Although Ayers and Donohue may very well have developed a “statistically superior model,” even they concede that “better” does not always mean “good enough.”[8] One possible interpretation of their data, they write, is that, “[w]hile the best evidence suggests that [concealed-carry] laws generally tend to increase crime, there is still too much uncertainty to make strong claims about their effects.”[9]

* * * * * 


[1] NRA-ILA, “Right-to-Carry 2012,” February 28, 2012.

[2] John Lott and David Mustard, “Crime, Deterrence, and Right-to-Carry Concealed Handguns,” Journal of Legal Studies, Volume 26, Number 1, January 1997.

[3] Ted Goertzel, “Myths of Murder and Multiple Regression,” The Skeptical Inquirer, Volume 26, Issue 1, January/February 2002.

[4] Whereas the Lott-Mustard model contained data from 1977-1992, the Ayers-Donohue model added data through 1999. This inclusion is significant since 14 new jurisdictions (13 states and the city of Philadelphia) added concealed-carry laws between 1992 and 1996.

[5] Ian Ayers and John J. Donohue III, “Shooting Down the ‘More Guns, Less Crime’ Hypothesis,” Stanford Law Review, Volume 55, Number 4, April 2003.

[6] It should be noted that when Ayers and Donohue plugged data from 1977 to 1999 into the Lott-Mustard model, they found that crime was (statistically) significantly higher in two categories and lower in two categories. They proceed to explain that the Lott-Mustard model yields lower crime rates because it contains “a large number of potentially duplicative demographic variables.” Specifically, Lott-Mustard contains “thirty-six separate demographic percentages, breaking down each of three different race categories—black, white, and neither black nor white—and both sexes into six separate age categories from age ten up.” Believing that “the array is so extensive as to make multicollinearity [that is, “the multiple counting of the same information”[6a]] a serious issue,” Ayers and Donohue reduced these demographic controls to twelve (six for whites above the age of ten and six for blacks) and after doing so found that “it is hard to find any crime category that seems to have a robustly lower crime rate.”

[6a] "Multicollinearity,"

[7] Ayers and Donohue write that “the dangers of estimating a single aggregated effect are particularly acute in this case because a state that adopts a shall-issue law early in the data period will contribute fully to the estimated postpassage effect, while a state that adopts near the end of the period will have little weight. Since we know that the late adopters tended to experience crime increases, the aggregated analysis will give less weight to these states in estimating the overall effects of shall-issue laws.”

[8] Another piece of evidence that the Lott-Mustard model is flawed: when Ayers and Donohue plugged 1977-1999 data into the model, they found that robbery did not decline in concealed-carry states. Because robbery is  committed in a public place more than any other crime,  it should be the crime most likely to decline if the Lott and Mustard story of deterrence has any plausibility.  Lott and Mustard agree with the general tenor of this assessment. As they write in their 1997 article: “Generally, we expect that the crimes most likely to be deterred by concealed handgun laws are those involving direct contact between the victim and the criminal, especially those occurring in a place where victims otherwise would not be allowed to carry firearms. For example, aggravated assault, murder, robbery, and rape seem most likely to fit both conditions.  

[9] For more on this agnostic view, see Chapter Six of the National Academies’ 2004 work, Firearms and Violence: A Critical Review.