Academic journals and journalists both perpetuate misinformation about academic research, a new study finds.
Journals can take months to years to retract unreliable research while journalists often fail to inform the public when the scientific discoveries they have reported on are later determined invalid or fraudulent.
The analysis reveals researchers play a role, too. Some researchers rely on and use discredited research in their work — even years after a journal has withdrawn the research from publication because problems such as scientific misconduct, significant errors, plagiarism or ethics violations have rendered those results untrustworthy.
Lead author Stylianos “Stelios” Serghiou explained the new study highlights key problems in the pipeline for communicating science to the public. In an email interview, he stressed the need to correct the scientific record as well as the public record following a retraction, “especially in the case of consequential or highly popularized articles.”
“When highly popularized research is retracted, we need to consider the obligation to inform the public that this research is no longer considered credible by the scientific community,” Serghiou, who has a medical degree and completed a doctoral degree in epidemiology and clinical research at Stanford University last year, wrote to The Journalist’s Resource.
The best way to do that, though, is unclear, Serghiou and his colleagues explain in their new paper. When they looked at how people discussed a small sample of retractions on social media, they discovered that, in some cases, “retraction was directly or indirectly associated with the promotion of the initial misinformation.”
Even though thousands of papers have been retracted over time, they remain rare. About 4 in 10,000 papers are retracted, Science magazine reported in 2018. Retraction Watch, a project of the nonprofit Center for Scientific Integrity, tracks retractions and has built a database of more than 26,000 retractions going back to 1756.
Serghiou and his colleagues examined 3,008 retracted papers published from Jan. 1, 2010 through Dec. 31, 2015 and their retraction notices, extracted from the Retraction Watch database.
To measure how much attention the papers received — before and after their retractions — from news outlets, social media and other sources, Serghiou and his fellow researchers obtained data from Altmetric, which tracks mentions of published scholarly literature across the internet. The researchers gathered data from Crossref on how often and when retracted papers were cited in other research.
The resulting paper, “Media and Social Media Attention to Retracted Articles According to Altmetric,” was published this month in the open-access journal PLOS ONE.
Some other takeaways:
- Most papers and their retractions tend to draw similar amounts of attention from news outlets, social media and other sources. But the most popular papers received 2.5 times as much attention before they were retracted than afterward.
- A median 457 days passed between a paper’s publication and its retraction.
- Of the retracted papers examined, 31 were cited more than 100 times each in other research.
- The most common reason for retraction during the study period was duplication, when a researcher or group of researchers publish the same information in two different papers. The second most common was “fake peer review,” meaning the authors cheated — intentionally or unintentionally — the peer-review process through which scholars vet and evaluate one another’s work. Plagiarism was the third most common reason.
- Tumor Biology had the largest retraction during the study period. In 2017, the journal published a retraction note withdrawing 103 articles due to fake peer review.
How people discuss retracted research on Twitter
Serghiou explained that journalists should report on retractions to correct the public record, but it’s unclear the best way to do that. He told JR there could be unintended consequences when journalists correct the record — potentially spurring social media users to promote the retracted findings as valid.
One retracted paper they studied, published in JAMA Pediatrics in 2012, claimed that putting an Elmo sticker on an apple encouraged children to choose apples over cookies. Before its retraction, people tweeted about the paper four times. The retraction, Serghiou noted, led to a flurry of tweets promoting the invalid finding. Someone tweeted a sentence suggesting stickers make children choose fruit over cookies and it was retweeted 742 times.
“In our limited study of tweets about recent popular retracted articles, we were very surprised to witness that retraction without appropriate communication may lead to inadvertent promotion of the results being retracted,” Serghiou wrote to JR.
Serghiou’s two co-authors are Rebecca Marton, a senior scientific researcher at the biotechnology company Genentech, and John Ioannidis, a Stanford University professor of medicine, statistics and health research widely known for his studies on the replicability and reliability of academic research. Ioannidis’ essay, “Why Most Published Research Findings Are False,” which appeared in PLOS Medicine in 2005, has been cited nearly 10,000 times.
Expediting retractions
Serghiou said helping the public better understand the scientific process, including the uncertainty and limitations in new findings, could lead to faster retractions of invalid papers. People need to know there are various reasons journals publish retractions, he explained.
“We need to consider that not all retractions are the product of premeditated intent to deceive, and that such connotations have led to substantial delays in correcting the scientific record,” he noted.
In their paper, Serghiou, Marton and Ioannidis stress the importance of clear and informative retraction notices and putting warning labels on retracted papers. They point out that some journals do not comply with guidelines for handling retractions established by the nonprofit Committee on Publication Ethics.
They cite a study in BMC Research Notes from 2013. The study reveals 9% of the 233 retractions published in 2008 found through Medline, the National Library of Medicine’s bibliographic database of research in the life sciences, did not include a detailed reason for the retraction. Also, 22% of those retracted papers were available with no mention of the retraction on the paper itself or on the journal website.
Ivan Oransky, who co-founded Retraction Watch, says journal publishers need to investigate and withdraw problematic papers much more quickly. He adds that journalists must do their part by educating themselves about journal publication and the limitations of peer review, the formal process through which researchers evaluate and provide feedback on one another’s work.
Oransky and Alison Abritis, a Retraction Watch researcher, were among the experts PLOS ONE selected to peer review the paper by Serghiou, Marton and Ioannidis.
Oransky, who teaches medical journalism at New York University, urges journalists to remember knowledge is provisional.
“Know that the amazing, wonderful, exciting findings that you wrote a story about could end up being retracted,” he recommends. “If you realize and keep in mind when you’re reporting on something that you’re reporting on a tiny bit of the truth — a tiny part of it — it makes you a bit more humble. If you’re reporting on a single study, any time there’s a retraction, it means your story is unraveling.”
4 tips for journalists
There are several things journalists can do to track flawed research and avoid spreading misinformation. Here are four tips based on the new paper and interviews with Oransky and Serghiou:
- Take advantage of technologies that flag retractions. For example, Zotero, a free tool for collecting and organizing research, automatically checks its users’ databases and papers for retractions. Zotero began offering the feature in 2019 through a collaboration with Retraction Watch. PubMed, a federal government database of primarily biomedical and health research, “has been flagging retractions for years and it recently introduced a larger banner to help make retractions more apparent,” Serghiou, Marton and Ioannidis write.
- Keep an eye on discussions happening on PubPeer, an online platform operated by the nonprofit PubPeer Foundation. Scholars and others use it to weigh in on published research and some conversations have prompted retractions, Oransky says.
- Focus on the findings of groups of studies rather than individual studies. “With groups of studies,” Oransky says, “the chance of a retraction destroying the whole thesis is negligible.”
- Check Retraction Watch for new retractions. Use its searchable online database to identify trends, such as a spike in retractions on a particular topic or from a specific researcher or institution.
Expert Commentary