How Was the “98% Success Number” for Bear Spray Computed?

One of the most quoted and controversial figures used in the debate about how effective bear spray is “98%”.  The figure is sprayed about (pun intended) promiscuously and irresponsibly.  From thetrek.com, quoting Tom Smith:

Ninety-eight percent of people who used bear spray escaped injury (with 2% being knocked over but not killed) as opposed to only about 50% of people using guns.

The number traces back to a paper authored by Tom Smith and Stephen Herrero, published in 2008. The paper is “Efficacy of Bear Deterrent Spray in Alaska“. It is the seminal paper that launched bear spray as a supposedly credible deterrent to stop aggressive bears.  In the 72 incidents used in the paper, only one involved a hunter. He was stalking a wounded bear. 30% were people involved in bear management activities. Aggressive bears accounted for 25 incidents or 35%. There were ten incidents where bears charged people or 14%.  All three injuries occurred in the ten incidents where bears charged people.  If we consider the bear spray success rate only for aggressive bears and count injuries as failures,

….the success rate for aggressive bears is 88%.

There were 175 people involved in the 71 incidents. The incident which was subtracted from the 72 is not explained.  The 98% figure is arrived at by dividing the number of people injured by the number of people involved. The paper does not mention how many of the 175 people had bear spray or sprayed bear spray. We know bear spray was sprayed in all 72 instances.  In ten of the seventy one incidents, bear spray had negative effects on the person using the spray.  Six of the 71 instances were deemed a failure to stop the undesired behavior of the bears. In 13 cases, the bears resumed the undesired behavior after being sprayed. The paper does not tell us if the three injuries were included in the bear spray failures or not. From “Efficacy of Bear Spray.” [embedded below]

Three persons (< 2% of the 175 persons involved in 71 separate incidents) suffered injury by bears that had been sprayed with bear deterrent.

The precise circumstances of the use of bear spray are not revealed for most of the incidents. An exception exists for polar bears.  From the paper:

 In both polar bear incidents, subadult bears approached humans in a pickup truck there to observe bears feeding on bowhead whale (Balaena mysticetus) remains near the village of Kaktovik, Barter Island, Alaska, USA. In both instances (100%), bear spray stopped the bear’s approach and turned the bear away. Neither of these bears returned to the truck following spraying.

Why the authors included these polar bear incidents as examples of where people sprayed menacing bears is unclear. They were not counted as aggressive bears.

In 2018, the authors updated their numbers through 2015. In the six years of 2010-2015, they found only four instances where bear spray was used. Those instances involved another 22 people.

As of 2015, 75 instances of bear spray use were recorded, of which 70 (93.3%) were successful in altering bears’ aggressive behavior, whereas 5 (6.7%) were not. However, of the 197 persons involved in these 75 encounters, only 4 received slight injuries (2.0%), all inflicted by grizzly bears.

This kept the ratio of people injured to total people involved at 2%. We are not told how many of the 197 actually had bear spray or used the bear spray.  Author Tom Smith informed this writer all 71 instances from the previous bear spray study were included in the 75 incidents.

“Efficacy”: Is defined as the ability, especially of a medicine or a method of achieving something, to produce the intended result.

The study most compared to “Efficacy of bear spray”  is “Efficacy of Firearms,” published in 2012. Tom Smith and Stephen Herrero were the authors of both studies.

In 2019, Tom Smith said there was no thought of comparing the studies:

“There was no thought of comparing the two [studies], though some do that,” says Tom Smith, who authored both reports, titled the “Efficacy of Bear Deterrent Spray in Alaska” and “Efficacy of Firearms for Bear Deterrence in Alaska.”

Direct comparisons are made to the “Efficacy of firearms,” but different methods are used for computations in the two papers. The samples are completely different. From a critique of the studies:

 “Comparing the two studies is like comparing the injury rate for people picking up apples to the injury rate for people picking up live hand grenades,” says Dave Smith, a naturalist who has worked in Yellowstone, Glacier, Denali, and Glacier Bay National Parks and who has authored two books on surviving dangerous encounters with wildlife.

In the Efficacy of Firearms study, there were 269 total firearm incidents. The authors subjectively picked the incidents. 215 of them involved aggressive bears. The success rate for stopping aggressive bears was 75% for long guns and 84% for handguns. In 15% of the incidents involving firearms, the firearms were not fired.

In the bear spray study, only incidents where bear spray was sprayed were included?

If the 15% of cases where firearms were not fired are removed from the 215 cases involving aggressive bears, the success rate increases to 89% for long guns and 99% for handguns, assuming the failures to fire are divided proportionately. This is speculation because the circumstances of the cases are not given. The data has not been released to the public or other researchers. We only know how the authors graded the cases.

The problems with access to bear spray and access to handguns are very similar. Handguns are more ergonomic than bear spray.  In this writer’s study of incidents involving handguns, the success rate is 98% for 170 incidents (146 handguns only).

The supplemental data for the Efficacy in Firearms study indicates no one carrying a handgun was killed.

If we look at the raw numbers and ignore the enormous difference in aggressive vs. non-aggressive bear samples, the total number of people involved in the 269 firearms cases was 444. The total number of injured (including killed) was 122. From the supplemental data in the Efficacy of Firearms study:

 Bear-inflicted injuries occurred in 151 of 269 (56%) incidents. Of the 444 people involved in firearms incidents, 122 (28%) were injured:

Math nerd note: 122/444 is .2748, or properly rounded, 27%.

This writer asked author Tom Smith to explain the apparent conflict between the 151 incidents involving injuries and the 122 people injured.

There may be a simple explanation. Tom Smith replied he did not know but he will be looking into it. In Tom’s defense, the paper is less than clear about how all the numbers are arrived at. A table of all the incidents, the dates they occurred, number of people and bears involved would have been very helpful. Unfortunately, the basic data used in the paper have not been made available to other researchers or the public.

Using the peculiar math of the bear spray study, 73% of the people involved in defenses involving firearms were not injured. The number would be much higher if cases, where firearms were not fired were excluded. This is a far cry from the 50% number cited in the comparison of the two studies.

It is not credible to compare the percentage of incidents with the percentage of people involved or to compare incidents where bear spray was used to incidents where firearms were not used.

Many more people have carried and used bear spray since 2015. Of the total number of incidents where handguns were recorded as used in defense against bears (170), one person was killed from 1890 to 2022. In comparison, in incidents where bear spray was used (number unknown), seven people have been killed from 2003 to 2022.

It is impossible to know how often long guns, handguns, or bear spray are used in defense against bears.  It is irresponsible to claim “science” in comparing studies, which use different definitions and math, to say firearms or bear spray is more effective.  Failures of defensive methods are recorded far more often than successes.

Efficacy of Bear Deterrent Spray in Alaska

About Dean Weingarten:

Dean Weingarten has been a peace officer, a military officer, was on the University of Wisconsin Pistol Team for four years, and was first certified to teach firearms safety in 1973. He taught Arizona concealed carry course for fifteen years until the goal of Constitutional Carry was attained. He has degrees in meteorology and mining engineering and retired from the Department of Defense after a 30-year career in Army Research, Development, Testing, and Evaluation.