From the Scholars Strategy Network, written by Debra Javeline, University of Notre Dame.

 

Scientific information is increasingly relevant to policy on health care, unemployment, climate change and other major issues, yet the quantity of available and accessible published research is often insufficient for effective policymaking. At such times, policymakers may consult scientists themselves for interpretations of the evidence to date. How can legislators and other public officials get an honest representation of what scientists know? How can citizens be sure their representatives in government are acting on the best available scientific knowledge? Current methods for engaging science in policymaking are flawed. My research along with colleagues suggests that standardized methods for aggregating scientists’ opinions can be a useful tool for policymakers to understand the state of scientific opinion on major public issues.

A broken system for citing science

Currently, policymakers often promote scientific opinion that happens to reinforce preexisting preferences and ignore, downplay, or dismiss contrary scientific findings. Policymakers also often claim that scientific opinion is divided, even when disagreement is actually minimal. Ammunition for dubious claims about science comes from industry researchers and partisan think tanks, which are proliferating. The media’s commitment to staging debates with “both sides” given equal time can mistakenly suggest deep divisions in the scientific community on issues where overwhelming consensus actually exists.

Some public servants avoid the noise and try earnestly to understand scientific debates, take account of uncertainty and assess the preponderance of evidence. But even such well-intentioned public servants do not follow a standardized approach. There are no guidelines for whose testimony should best represent the state of science, so well-connected or famous individual scientists are often invited to interpret the current state of findings in their fields. Policymakers rarely know if the judgments of these selected experts reflect the views of the majority of equally expert scientists. The result can be a battle of “our experts versus your experts,” with no metrics for gauging who is right.

Standardizing portrayals of scientific opinion

There will always be disagreements about how to act on scientific information, but there should be little room for debate on what the science actually is. For example, we now know with a reasonable degree of certainty that smoking increases the likelihood of cancer, leaving policymakers to debate whether and how government should act. But false claims of divided science about smoking and cancer delayed the relevant policy debates for decades, much as false claims today misrepresent the scientific consensus about human-caused climate change and delay policy debates and action. A standardized strategy for assessing the range of scientific opinion would allow policymakers to take science as a premise and consider remedies.

Such measurement can be accomplished by defining scientific opinion as the aggregate opinions of the most informed experts in relevant fields. With the relevant scientists selected in objective, systematic, and replicable ways, scientific opinion can be surveyed in ways resembling the current best practices for measuring mass public opinion, leading to the following advantages:

• Standardized approaches allow portraits of both majority and minority opinions among scientists, ensuring that dissent, uncertainty and near-consensus are visible.
• Standardized measures can be repeated over time, to allow for the monitoring of shifts in scientific opinion as new evidence is accumulated and experts update their assessments.
• Minority views can be shown to grow or dwindle over time if scientists shift their opinions in response to new and more convincing data.

Exactly how should scientific opinions be measured? Viable approaches include canvassing published peer-reviewed articles and perhaps non-peer-reviewed sources like scientific reports, statements from professional organizations and multi-signatory statements designed to inform policy. This approach is likely to be most fruitful for mature scientific fields where reported findings and expert opinion converge and may become identical.

For less mature scientific fields with fewer published articles, another measurement approach involves opinion surveys of a population or sample of scientists who can offer the best interpretation of science-in-progress. This involves defining a target population of scientists, creating a list and conducting a census or a sample survey of the relevant experts.

How better measurement can improve public policy

My colleagues and I tested our approach on a current environmental issue. To save endangered species from climate-induced extinction, some people have proposed “managed relocation” to new, more climatically suitable regions. The idea remains controversial, and research is in its infancy. To measure the current range of scientific opinions, we surveyed environmental biologists who had published in the top quartile of ecology journals over the most recent five years, asking 2,329 experts whether and under what circumstances managed relocation should be legal or encouraged. Our results not only gave an overview of scientific opinion but also offered novel insights. For example, respondents offered specific guidance on the circumstances under which managed relocation might — or might not — be efficacious.

The recommended methods can readily be applied to other scientific issues. To the extent that policymakers cite scientific opinion, standardized measurements can ensure that policymakers start with a shared premise about science and then focus their debates on policy choices. Citizens can also benefit, because elected representatives would have to offer honest accounts of decisions, rather than misrepresenting science or falsely pointing to “divided” experts. In short, rigorous measurement can allow honest policymakers to respond to accurately portrayed science, leading to better informed policies and improved democratic politics.

Related research: In the article “Scientific Opinion in Policymaking: The Case of Climate Change Adaptation,” published in Policy Sciences, Javeline and Gregory Shufeldt look at ways that scientific opinion can be measured. These include “systematic surveys of scientists that employ standards similar to those that govern public opinion surveys, including systematic decisions about target populations, sampling frames, and sampling techniques.”

 

The author is a member of the Scholars Strategy Network, where this post originally appeared.

Keywords: policymaking, science, scientific consensus, research brief