Scientific Judgment , USFWS and the Fate of Yellowstone’s Grizzlies
The grizzly bear of Yellowstone is expected to be delisted from the Endangered Species Act. But a survey of grizzly bear researchers finds flaws in how wildlife experts evaluate scientific data.
In March, the U.S. Fish and Wildlife Service (USFWS) announced its intent to remove protections afforded by the U.S. Endangered Species Act (ESA) to grizzly bears in the Greater Yellowstone Ecosystem (GYE).
Citing four decades of growth in the bear population, the USFWS Director Dan Ashe heralded the decision as “a historic success for partnership-driven wildlife conservation .”
However, conservation organizations oppose “delisting” GYE grizzlies. They cite persistent threats to grizzlies, public opposition to delisting and ongoing scientific uncertainty regarding the population’s viability. Indeed, scientific uncertainty, especially threats posed by a changing climate, is one reason a federal court reversed a similar decision back in 2009, returning federal protections to GYE grizzlies.
According to the ESA, decisions about the listing of species are to be made “solely on the basis of the best scientific and commercial data available.” The ESA also mandates that decisions about whether to list a species be guided by a scientific assessment of potential threats to a species.
The ESA is hardly unique in this regard. Numerous environmental statutes mandate that government agencies consider “the best available science” when making decisions. And agencies routinely consult with scientific experts to fullfill such mandates.
These requirements ostensibly insulate decisions from undue political influence. Such provisions work reasonably well when science offers clear and simple, black-and-white answers. But when there is uncertainty, is the expectation of scientific objectivity realistic?
To gain insight into what role bias may play in listing decisions, we surveyed a group of grizzly bear researchers. We found that experts' judgments were associated with a number of factors outside the “best commercial and scientific data,” including their professional affiliations and social norms. Furthermore, we found that while there is no consensus in the scientific community regarding the threats to grizzly bears, the majority of scientists support continued listing.
The Yellowstone grizzly: a case study in scientific uncertainty
In December of 2014, we (along with graduate student Harmony Szarek and Dr. Eric Toman ) contacted 593 individuals who published research related to grizzly or brown bears during the prior decade. We asked them to judge the risk (likelihood and severity) of seven threats identified by the USFWS and to recommend the appropriate conservation status – delisted, threatened or endangered – for GYE grizzlies.
In total, 60 percent of 211 respondents recommended continued protection under the ESA, about one-fifth indicated the population should be delisted and a similar proportion were unsure. When unsure respondents were removed, 74 percent of 172 experts recommended continuing ESA protections (see project report ).
As expected, we found considerable evidence of uncertainty in experts’ ratings of risk (indicated by variability across experts). But surprisingly, judgments about the conservation status of grizzlies were unrelated to level of expertise. Rather, conservation judgments were strongly associated with the type of organization that employed the experts.
We found those working for state or federal wildlife agencies were 2-3 times more likely to recommend delisting grizzlies than those employed by academic institutions. Furthermore, experts belonging to The Wildlife Society , which has openly advocated delisting bears since 2011, were more likely than members of other professional organizations (e.g., The American Society of Mammologists ) to recommend delisting.
Our data suggest that conservation judgments were influenced not so much by an expert’s knowledge or assessment of risk but more so by their social environment; in particular, the peers with whom an expert regularly interacts and respects.
That interpretation is reinforced by additional analyses indicating that experts' judgments about bears' conservation status were statistically associated with their beliefs about (i) the listing judgments of other scientists and managers (i.e., their “social norms”), and (ii) the way humans ought to relate to wild animals – what researchers call “wildlife value orientations.” Further, when controlling for these factors, judgments about the conservation status of grizzlies were not statistically associated with experts’ assessments of risk.
Social and political pressure
Psychologists have long known that social pressure created by the presence of others can influence our thoughts and actions. A now-classic experiment shows that people will even give obviously wrong answers to empirical questions when under the sway of the group. But our experts faced no such pressure. Their responses were given anonymously.
Our data suggest that social pressure can affect judgments even when we’re alone. Of course, while the norms of the social groups we interact with are likely to shape our judgments, we are also likely to select social groups that reflect the norms we embrace. In either case, our concern is that supposed scientific judgments may well be heavily influenced by socially segregated groups and their associated beliefs.
Of course, it is not inherently problematic that an expert’s judgment is affected, in part, by how he or she expects respected peers would judge a given circumstance. Nor is it necessarily problematic that judgments about conservation routinely depend on factors beyond science, like one’s values and emotions . Indeed, the dichotomy between facts and values may well be a false dichotomy, as argued by the great American philosopher Hillary Putnam .
It is, however, potentially problematic when groups of scientists disagree about the appropriate status of a species. Especially when the individuals charged with providing guidance on species recovery are comprised entirely of one of these groups – in the case of GYE grizzly bears, members of state and federal agencies who were 2-3 times more likely to advocate delisting.
What’s concerning here is that, as opposed to academic scientists who are somewhat shielded from politics by tenure, scientists in state and federal agencies can face strong, top-down pressure to reach a particular decision.
Expert judgment and ‘the best available science’
Is there a way to apply the best available science to conservation decisions in order to minimize the effect of undue biases and, to the extent practicable, isolate scientific judgment from other factors? Two sources of scholarly research provide useful insights.
First, scholars in the field of judgment and decision-making have long recognized the problem of implicit bias, noting the often substantial gap between how we would ideally think and how we actually think. These scholars have developed a variety of strategies for assisting people in “ debiasing ” judgments and decisions.
These strategies can be remarkably simple. Often the best technique is simply being aware of one’s own biases and predispositions, and engaging in “perspective taking” to assess if one’s own judgments are unduly influenced by preexisting values and beliefs.
Other strategies are far more involved, and considerably more labor-intensive. For example, Structured Decision-Making (SDM) is an approach for identifying and evaluating alternatives through a process designed to deal with complexity, value-based trade-offs and competing interests. This process focuses on clearly defining the problem at hand and identifying the full suite of decision-relevant objectives, such that the best available science can be used to evaluate the utility of different actions and the trade-offs in choosing one option over another.
Although such approaches can help decision-makers negotiate uncertainty, their appropriateness depends greatly on their application. One concern is that they can be readily manipulated to obtain a prejudged outcome. An especially pernicious manipulation is the purposeful exclusion of some interests, thereby limiting participants to like-minded individuals and constraining the range of values taken into account. Such concerns lie at the heart of a recent report criticizing the USFWS’s use of SDM to develop a National Wolf Strategy.
A second source of underappreciated insights is from the academic discipline of conservation ethics. A broadly applicable insight from that discipline is that robust conservation decisions result from sound and valid arguments that are necessarily comprised both of scientific premises and ethical premises.
The implication is that in many instances the best an expert can do is explain their judgment fully. That is, to lay bare all of the premises (scientific and otherwise) necessary to arrive at the judgment being proffered; and in so doing, demonstrate the robustness of the judgment (or reveal its flaws).
So an expert judgment is not merely a judgment made within the area of one’s expertise. Rather, an expert judgment is one whose underlying argument can be laid bare and demonstrated to be sound and valid for an audience of nonexperts. Importantly, this includes both scientific assessments as well as value judgments. Sadly, courses that convey skills in analyzing ethical arguments (i.e., courses in critical thinking and environmental ethics) are not typically part of the curricula that produce conservation professionals.
Our research should not be taken as an unqualified condemnation of the USFWS’s decision to delist GYE grizzlies, though the fact that nearly three-fourths of experts felt the species should remain listed should give the USFWS pause. Rather, our interest here is to call attention to potential sources of bias, and call on the agencies charged with using the best available science to take efforts to minimize bias to the extent practicable.
We recognize this is likely to prove a challenging task. Fortunately, agencies need not brave that challenge alone. Indeed, the first step in minimizing bias is broadening the type of experts with which agencies consult to include scholars well-versed in these tools.