On April 30, 2018 the U.S. Environmental Protection Agency came out with a proposed rule titled “Strengthening Transparency in Regulatory Science.”
The new rule would limit the kinds of scientific studies the agency could use in support of future regulations. In short, in order for EPA to take a study’s finding into account when developing a regulation, the researchers would need to have made all the data in that study available to the public.
“The proposed regulation provides that when EPA develops regulations, including regulations for which the public is likely to bear the cost of compliance, with regard to those scientific studies that are pivotal to the action being taken, EPA should ensure that the data underlying those are publicly available in a manner sufficient for independent validation,” the summary of the proposed rule states. “This proposed rule is designed to increase transparency in the preparation, identification, and use of science in policymaking.”
While gaining access to data may sound appealing — especially to journalists — the policy could end up discounting many high-quality studies from EPA rulemaking.
Veena Singla, associate director of science and policy at the University of California, San Francisco’s Program on Reproductive Health and the Environment, explains that many studies rely on confidential data — for example, health studies involving human subjects.
Generally, institutional review boards — independent ethics committees housed at research institutions — mandate that data collected from human subjects remain confidential.
The proposed rule has faced strong opposition from scientists.
It’s one thing to make a study available to the public, but quite another to make all the related data publicly available, Singla says.
“What this rule is going after is actual, individual measurements,” Singla explains. “So, for example, a study might publish the results of average blood lead levels of this population of children that we looked at. The study will publish the number of children in the study, all kinds of information about the data in aggregate. And what the EPA will want is an individual child measurement, along with all the information about that individual child that the researchers used to calculate anything in their study. That’s pretty different … having all of that individual-level data publicly available is very different than having the study and its results publicly available.”
Journalist’s Resource interviewed Singla about a recent panel discussion she participated in during the American Public Health Association’s annual meeting in Philadelphia.
For additional insights, we also spoke with Joseph Goffman, a former EPA administrator who is now the executive director of Harvard Law School’s Environmental & Energy Law Program. During his tenure at EPA, Goffman was a chief architect of the Clean Power Plan of 2015. He helped craft many other rules, including the Mercury and Air Toxics Standards and the Cross State Air Pollution Rule, both established in 2011.
Singla and Goffman offered expert tips for understanding and reporting on the proposed rule along with other key changes occurring at EPA. Both consider “Strengthening Transparency in Regulatory Science” a misleading title.
1: Note the difference between making a study available to the public and making all the data publicly available.
Academic debates over open access scholarship are not the issue here, Singla notes. “It’s about the underlying individual data points that went into the study,” she says.
2: Explain to audiences how the title of a proposed rule might mislead.
“The conundrum that reporters have is that the agency gets to choose its own titles,” Goffman says. “And up until this point, it’s been a universally observed convention that everybody who talks about a particular rule or proposal goes with the caption or title that the agency chose. A more accurate title would be, this is a ‘Narrowing Rule.’ Instead of a transparency-in-science rule, this is a narrowing science rule.”
Adds Singla: “It’s very contradictory because ‘transparency’ sounds like a good thing. The evidence and facts show that more transparency is not going to be the outcome of this policy approach. This is not going to be effective at increasing transparency. The end result is going to be, actually, limiting science. I think that’s the crux of it is [journalists] being able to explain why and how this rule is not going to achieve that result.”
For example, in an editorial published in PLoS Medicine, John P.A. Ioannidis, a professor and co-director of the Meta-Research Innovation Center at Stanford University, highlights potential effects of the rule. “In a random sample of the biomedical literature (2000–2014), none of 268 papers shared all of their raw data,” he writes. “Only one shared a full research protocol. If the proposed rule is approved, science will be practically eliminated from all decision-making processes. Regulation would then depend uniquely on opinion and whim.”
3: Find out how far along the rule is in the administrative rulemaking process.
EPA accepted public comments on the proposed rule between April 30, 2018 and Aug. 16, 2018. The agency received over 590,000 comments and is reviewing them. The next step in the rulemaking process is to issue a supplemental proposal to the rule. A supplemental proposal is essentially a revised draft of a rule. EPA had indicated the supplemental proposal would be issued in January 2020; however, to date, the draft is still in process.
“The agency rulemaking process goes through many, many steps,” Goffman explains. “And the steps often take the form of drafts or a succession of discrete, revised drafts of preambles and regulatory text. And that goes on. You know, usually those texts fall under the pen — either the drafting or editorial pen — of maybe two dozen people in the agency, located in different offices, including, but not limited to, the General Counsel’s Office. And so at any one time, any one document may or may not be representative of the document that ultimately goes out of the agency, formally, to the Office of Management and Budget for review.”
On Nov. 11, 2019, The New York Times published a leaked version of the supplemental proposal. The following day, the EPA’s press office issued a statement saying the accompanying article “is based on leaked preliminary, draft documents that are not accurate and do not include the final text submitted to the Office of Management and Budget (OMB) for interagency review.”
“This was a necessarily incomplete document in the way that a snapshot of a moving object is incomplete,” Goffman says. “But it doesn’t mean that the snapshot was inaccurate. In its typical, disingenuous fashion, the agency misused the term, [saying] that the Times’ reporting was inaccurate because the Times said, “Here’s what the document we got looks like.”’
In its statement, the EPA says the agency intends to issue a final rule in 2020. What that means, procedurally, is that after the supplemental proposal is released, there will be a public comment period, and then the EPA will review and respond to substantive comments before finalizing the rule.
“If they do indeed finalize the rule in 2020, they would have needed to respond to all of those substantive public comments in an adequate way,” Singla says. “And if they don’t, that can be the basis for legal challenges. In terms of the process, those next steps are important for journalists to understand.”
While the supplemental proposal hasn’t been released, Singla says witness testimony from Dr. Jennifer Orme-Zavaleta, principal deputy assistant administrator for science in the EPA’s Office of Research and Development (ORD), offers hints of what the supplemental proposal might entail. Orme-Zavaleta answered questions during the House Committee on Science, Space and Technology’s hearing on the rule on Nov. 13.
A key question, and an issue raised by The New York Times’ reporting, was whether the rule will apply retroactively.
“Her answer to that was ‘no’ and ‘yes,’ in a way,” Singla explains.
Orme-Zavaleta indicated that the policy will not apply to existing rules and regulations, only prospective, or new, regulation. However, when past rules and regulations come under review, all science involved will be subject to the “transparency” rule.
Singla continues, “Past science would be subject to the transparency rule as it comes up during any new rulemaking, and any review of existing standards or rules. So, like I said, it’s a no and a yes, and the yes portion of it is concerning because there are ongoing reviews mandated under many laws, and that would open up any of those past decisions and the past science to the rule.”
4: When deciding whether to report on leaked documents, consider why they were leaked.
“In seven years-plus that I worked at the agency, the many, many rules and comparable documents that I worked on, I don’t remember a non-final version draft leaking,” Goffman says. “I believe that the agency was managed by the politicals such that there was a kind of unquestioned mutual trust between the experts and the political managers. When I see a document like this leaking, the first thing I hypothesize is that there was not only a substantive disagreement between the political leadership and the expert staff, but that whoever leaked it must have had a reason not to trust that the subsequent process would be an honest one.”
Other EPA issues to watch for:
While the proposed “transparency” rule is getting a lot of media attention, there are many EPA issues getting less attention that are worth keeping an eye on.
“There’s a broader point. I think the political leadership, and the coalition they represent, have a good understanding of their own project, in the sense that they don’t just want to roll back a series of individual rules. They really want to decommission important components of the regulatory apparatus itself,” Goffman says. “And so it’s well advised for journalists, and other interested parties, to be attentive to parallels and patterns in these different enterprises.”
Singla offers an example: the EPA’s Toxic Substances Control Act (TSCA) systematic review method.
“The US Environmental Protection Agency (EPA) is responsible for making evidence-based policies to limit exposure to dangerous chemicals,” Singla writes in a commentary published in the American Journal of Public Health. “To inform potential chemical regulations, a core component of the EPA’s duty is to evaluate data on the hazards and risks of industrial chemicals under the 1976 Toxic Substances Control Act. Congress reformed the TSCA after widespread recognition of fatal flaws in the 1976 law. Under it, the EPA could not even restrict asbestos, a known human carcinogen. In 2016, President Barack Obama signed the Frank R. Lautenberg Chemical Safety for the 21st Century Act (Pub L No. 114-182), overhauling TSCA after 40 years.”
The overhaul includes a new methodology for collecting, evaluating and interpreting scientific evidence about chemicals.
“The EPA officially calls the method a ‘systematic review’ framework for TSCA, but it is systematic in name only, as it falls far short of best practices for systematic reviews,” Singla writes. “Application of the TSCA method will exclude relevant research from chemical assessments, leading to underestimation of health risks and resulting in inadequate policies that allow unsafe chemical exposures, thus harming public health.”
A main concern is that it lacks key elements of a systematic review such as a pre-established protocol for the review prior to conducting it. Having a pre-established protocol would “minimize bias and ensure transparency in decision making,” Singla writes. Another concern is that the EPA uses a flawed approach to evaluate study quality.
“The TSCA systematic review method is a lot more arcane, in the weeds,” Singla says. “It hasn’t gotten that much attention, but it has the potential to do the same thing as this ‘transparency’ rule, which is ultimately to limit the science and information that the agency is relying on.”
In addition to TSCA, Singla suggests journalists keep an eye on the EPA’s efforts to revise its guidelines for cost-benefit analysis and risk assessment.
“What Administrator [Andrew R.] Wheeler has said is that they’re going to go back, statute by statute, starting with the Clean Air Act, to revise the way that EPA looks at benefits and costs,” Sigla says. “So again, this is one step back from particular rules and regulations, trying to change the fundamental way that the agency conducts its evaluations and analyses, to really bias them or stack them in a certain way, such as by limiting the kinds of evidence that’s used or changing the way that the agency considers the benefits of reducing pollutants.”
Goffman agrees: “What they started to try to do with their cost-benefit analysis represents a similar project, of weakening the impetus to regulate, or weakening the development of analysis that then triggers a statutory obligation to do additional regulation.”
Another issue to watch: Potential changes to the EPA’s cancer risk assessment guidelines.
“There’s also an effort underway for EPA to revise some of its risk assessment guidelines,” Singla says. “That’s looking at the risk of a chemical or pollutant — what are the health effects — and looking at the amount of the pollutant in the environment, or the amount that people eat, breathe or take in, [and] what are the health risks to people. That’s all done through EPA’s risk assessment process, and the agency has very clear guidelines on how chemical health effects are considered within that evaluation of risk, for cancer-causing chemicals and pollutants specifically. And there is an effort underway to revise those risk assessment guidelines and change the way that the agency is considering cancer-causing agents.”
She continues, “That’s something for journalists to watch, because it will change, across the board, the way that the agency looks at cancer-causing substances. It’s not related to any one specific chemical or substance, at this time. But it’s the underlying way the agency would look at the science, evaluate it and then come up with the risk. And the kinds of revisions they want to do would end up resulting in saying something’s less risky.”
Also from the APHA annual meeting: Research-based tips for reporting on science research. Also on Trump’s EPA: Formaldehyde and leukemia: What research reveals about the risks.
Expert Commentary