Amy Sacka, CC BY-ND
Frederik Joelving, Retraction Watch; Cyril Labbé, Université Grenoble Alpes (UGA), and Guillaume Cabanac, Institut de Recherche en Informatique de Toulouse
Over the past decade, furtive commercial entities around the world have industrialized the production, sale and dissemination of bogus scholarly research. These paper mills are profiting by undermining the literature that everyone from doctors to engineers rely on to make decisions about human lives.
It is exceedingly difficult to get a handle on exactly how big the problem is. About 55,000 scholarly papers have been retracted to date, for a variety of reasons, but scientists and companies who screen the scientific literature for telltale signs of fraud estimate that there are many more fake papers circulating – possibly as many as several hundred thousand. This fake research can confound legitimate researchers who must wade through dense equations, evidence, images and methodologies, only to find that they were made up.
Even when bogus papers are spotted – usually by amateur sleuths on their own time – academic journals are often slow to retract the papers, allowing the articles to taint what many consider sacrosanct: the vast global library of scholarly work that introduces new ideas, reviews and other research and discusses findings.
These fake papers are slowing research that has helped millions of people with lifesaving medicine and therapies, from cancer to COVID-19. Analysts’ data shows that fields related to cancer and medicine are particularly hard-hit, while areas such as philosophy and art are less affected.
To better understand the scope, ramifications and potential solutions of this metastasizing assault on science, we – a contributing editor at Retraction Watch, a website that reports on retractions of scientific papers and related topics, and two computer scientists at France’s Université Toulouse III–Paul Sabatier and Université Grenoble Alpes who specialize in detecting bogus publications – spent six months investigating paper mills.
Co-author Guillaume Cabanac also developed the Problematic Paper Screener, which filters 130 million new and old scholarly papers every week looking for nine types of clues that a paper might be fake or contain errors.
An obscure molecule
Frank Cackowski at Detroit’s Wayne State University was confused.
The oncologist was studying a sequence of chemical reactions in cells to see whether they could be a target for drugs against prostate cancer. A paper from 2018 in the American Journal of Cancer Research piqued his interest when he read that a little-known molecule called SNHG1 might interact with the chemical reactions he was exploring. He and fellow Wayne State researcher Steven Zielske began experiments but found no link.
Meanwhile, Zielske had grown suspicious of the paper. Two graphs showing results for different cell lines were identical, he noticed, which “would be like pouring water into two glasses with your eyes closed and the levels coming out exactly the same.” Another graph and a table in the article also inexplicably contained identical data.
Zielske described his misgivings in an anonymous post in 2020 at PubPeer, an online forum where many scientists report potential research misconduct, and also contacted the journal’s editor. The journal pulled the paper, citing “falsified materials and/or data.”
“Science is hard enough as it is if people are actually being genuine and trying to do real work,” said Cackowski, who also works at the Karmanos Cancer Institute in Michigan.
Amy Sacka, CC BY-ND
Legitimate academic journals evaluate papers before publication by having other researchers in the field carefully read them over. But this peer review process is far from perfect. Reviewers volunteer their time, typically assume research is real and so don’t look for fraud.
Some publishers may try to pick reviewers they deem more likely to accept papers, because rejecting a manuscript can mean losing out on thousands of dollars in publication fees.
Worse, some corrupt scientists form peer review rings. Paper mills may create fake peer reviewers. Others may bribe editors or plant agents on journal editorial boards.
An ‘absolutely huge’ problem
It’s unclear when paper mills began to operate at scale. The earliest suspected paper mill article retracted was published in 2004, according to the Retraction Watch database, which details retractions and is operated by The Center for Scientific Integrity, the parent nonprofit of Retraction Watch.
An analysis of 53,000 papers submitted to six publishers – but not necessarily published – found 2% to 46% suspect submissions across journals. The American publisher Wiley, which has retracted more than 11,300 articles and closed 19 heavily affected journals in its erstwhile Hindawi division, said its new paper mill detection tool flags up to 1 in 7 submissions.
As many as 2% of the several million scientific works published in 2022 were milled, according to Adam Day, who directs Clear Skies, a company in London that develops tools to spot fake papers. Some fields are worse than others: biology and medicine are closer to 3%, and some subfields, such as cancer, may be much larger, Day said.
The paper mill problem is “absolutely huge,” said Sabina Alam, director of Publishing Ethics and Integrity at Taylor & Francis, a major academic publisher. In 2019, none of the 175 ethics cases escalated to her team was about paper mills, Alam said. Ethics cases include submissions and already published papers. “We had almost 4,000 cases” in 2023, she said. “And half of those were paper mills.”
Jennifer Byrne, an Australian scientist who now heads up a research group to improve the reliability of medical research, testified at a July 2022 U.S. House of Representatives hearing that nearly 6% of 12,000 cancer research papers screened had errors that could signal paper mill involvement. Byrne shuttered her cancer research lab in 2017 because genes she had spent two decades researching and writing about became the target of fake papers.
In 2022, Byrne and colleagues, including two of us, found that suspect genetics research, despite not immediately affecting patient care, informs scientists’ work, including clinical trials. But publishers are often slow to retract tainted papers, even when alerted to obvious fraud. We found that 97% of the 712 problematic genetics research articles we identified remained uncorrected.
Potential solutions
The Cochrane Collaboration has a policy excluding suspect studies from its analyses of medical evidence and is developing a tool to spot problematic medical trials. And publishers have begun to share data and technologies among themselves to combat fraud, including image fraud.
Technology startups are also offering help. The website Argos, launched in September 2024 by Scitility, an alert service based in Sparks, Nevada, allows authors to check collaborators for retractions or misconduct. Morressier, a scientific conference and communications company in Berlin, offers research integrity tools. Paper-checking tools include Signals, by London-based Research Signals, and Clear Skies’ Papermill Alarm.
But Alam acknowledges that the fight against paper mills won’t be won as long as the booming demand for papers remains.
Today’s commercial publishing is part of the problem, Byrne said. Cleaning up the literature is a vast and expensive undertaking. “Either we have to monetize corrections such that publishers are paid for their work, or forget the publishers and do it ourselves,” she said.
There’s a fundamental bias in for-profit publishing: “We pay them for accepting papers,” said Bodo Stern, a former editor of the journal Cell and chief of Strategic Initiatives at Howard Hughes Medical Institute, a nonprofit research organization and funder in Chevy Chase, Maryland. With more than 50,000 journals on the market, bad papers shopped around long enough eventually find a home, Stern said.
To prevent this, we could stop paying journals for accepting papers and look at them as public utilities that serve a greater good. “We should pay for transparent and rigorous quality-control mechanisms,” he said.
Peer review, meanwhile, “should be recognized as a true scholarly product, just like the original article,” Stern said. And journals should make all peer-review reports publicly available, even for manuscripts they turn down.
This article is republished from The Conversation under a Creative Commons license. This is a condensed version. To learn more about how fraudsters around the globe use paper mills to enrich themselves and harm scientific research, read the full version.
Frederik Joelving, Contributing editor, Retraction Watch; Cyril Labbé, Professor of Computer Science, Université Grenoble Alpes (UGA), and Guillaume Cabanac, Professor of Computer Science, Institut de Recherche en Informatique de Toulouse
This article is republished from The Conversation under a Creative Commons license. Read the original article.
