Credibility of Science Journals Under Scrutiny - LV Guardian Express

Normally, the discovery of a potential new treatment for cancer would be considered good news; but when the study conducted on the substance is fake, and deeply flawed because it is meant to raise the eyebrows of, and be rejected by science journals, and it is still accepted for publication, the credibility of the peer review process comes under scrutiny.

In an article published by Science magazine, John Bohannon details how he sent deeply flawed studies with false data to 304 purported open access scientific journals. Not only that; but he sent them under the names of fictitious researchers at made-up universities in order to not give himself away by sending the same manuscript to all the journals he targeted in his investigation. He found that many of these journals were willing to publish his paper as long as he paid a fee. Some asked for minor changes, such as a different format or presentation of the data, longer abstracts, etc.; but did not criticize the bunk science behind the study.

Bohannon decided to proceed with this course of action after investigating one particular publisher, and finding out that one of its reviewers had only been asked to peruse one paper in the four years that she has been listed as being affiliated with the journal. Moreover, she asked for that study to be rejected for publication, and yet the manuscript was given the green light by the journal’s editorial staff. After that, the reviewer asked to have her name removed from the journal’s masthead, and yet, she is still listed as a reviewer for that journal.

Bohannon then wrote the paper he was going to use as bait, with huge flaws in the data he falsified, for example, the treatments that were being tested were only applied to cancerous cells, so that if they were also toxic to healthy cells, that would nullify their medicinal usefulness; however, the fictional study did not explore this possibility.

In order to not give himself away, Bohannon did not submit the same study to all the “scientific journals” he targeted. Instead, he replaced the names of different lichens, their extracts, different types of cancers, and different researcher and university names in each submission. Many feel as though his last step was unnecessary, really, as the publishers should have rejected the manuscript on the basis of the bunk science behind it alone in order to maintain their credibility. Also, a big red flag that calls for more scrutiny is that neither the researchers, nor the universities where they conducted the supposed studies exist, and a simple Google search would have revealed that fact.

When examining an issue such as this, we must explore its root causes as well, and one of the main ones, according to the interactive map that Bohannon’s investigation generated is that in India, in particular, professional scientists are under a lot of pressure to publish research in order to get coveted jobs or promotions, and most of these so-called journals that bypass the peer review process in exchange for money are based there.

So why does any of this matter? After all, why should we care about nerds reading research papers written by other nerds? It matters because the nerds supposed to be reading the papers are not actually reading them, and that leads to bunk science being accepted at face value by your insurance company, your doctor, and your lawmakers, even though in many cases it is deeply flawed. That is how charlatans can convince you, for example, that common vaccines are giving your children autism, a “fact” of which there is absolutely no convincing scientific proof, and which leads to outbreaks of disease that should long ago have been eradicated by now, such as the whooping cough outbreak that occurred in 2010 (the largest in California since 1947,) or the more recent measles outbreak in Texas just last month.

In addition, most of us have no use for such scientific studies in our daily lives; but snake oil salesmen constantly try to hijack the peer review process to try to sell us sham cures for people that are so desperately holding onto life that they are basically grasping at any straw offered to them. Also, scientific studies are constantly being used as guide rails for policy within the medical establishment and government. It behooves us, collectively, to make sure such academic papers are held to the same high standard as laws are, because they just might become law.

There are safeguards against such unscrupulous publishers, such as the Directory of Open Access Journals (DOAJ) started by Lars Bjørnshauge, a library scientist at Lund University in Sweden, which aims to list credible scientific journals, and a list compiled by Jeffrey Beall, library scientist at the University of Colorado, with the objective of discrediting scientific journals that aim to publish any studies without first reviewing them as long as the authors of the “scientific papers” can pay the corresponding fees.

Unfortunately, Bohannon’s investigation found that there is some overlap between those two lists. This should serve as a caveat to all persons with a critical mind that the peer review process is being compromised, and it is wise to scrutinize carefully everything one reads even if it is published in a “scientific journal” functioning under the thinnest veneer of credibility.

By Milton Ruiz

Source 1

Source 2

Source 3

Some Online Journals Will Publish Fake Science, For A Fee - NPR

Richard Knox

Many online journals are ready to publish bad research in exchange for a credit card number.

That's the conclusion of an elaborate sting carried out by Science, a leading mainline journal. The result should trouble doctors, patients, policymakers and anyone who has a stake in the integrity of science (and who doesn't?).

The business model of these "predatory publishers" is a scientific version of those phishes from Nigerians who want help transferring a few million dollars into your bank account.

To find out just how common predatory publishing is, Science contributor John Bohannon sent a deliberately faked research article 305 times to online journals. More than half the journals that supposedly reviewed the fake paper accepted it.

"This sting operation," Bohannan writes, reveals "the contours of an emerging Wild West in academic publishing."

Online scientific journals are springing up at a great rate. There are thousands out there. Many, such as PLoS One, are totally respectable. This "open access" model is making good science more accessible than ever before, without making users pay the hefty subscription fees of traditional print journals.

(It should be noted that Science is among these legacy print journals, charging subscription fees and putting much of its online content behind a pay wall.) 

But the Internet has also opened the door to clever imitators who collect fees from scientists eager to get published. "It's the equivalent of paying someone to publish your work on their blog," Bohannan tells Shots.

These sleazy journals often look legitimate. They bear titles like the American Journal of Polymer Science that closely resemble titles of respected journals. Their mastheads often contain the names of respectable-looking experts. But often it's all but impossible to tell who's really behind them or even where in the world they're located.

Bohannan says his experiment shows many of these online journals didn't notice fatal flaws in a paper that should be spotted by "anyone with more than high-school knowledge of chemistry." And in some cases, even when one of their reviewers pointed out mistakes, the journal accepted the paper anyway — and then asked for hundreds or thousands of dollars in publication fees from the author.

A journalist with an Oxford University PhD in molecular biology, Bohannan fabricated a paper purporting to discover a chemical extracted from lichen that kills cancer cells. Its authors were fake too — nonexistent researchers with African-sounding names based at the fictitious Wassee Institute of Medicine in Asmara, a city in Eritrea.

With help from collaborators at Harvard, Bohannan made the paper look as science-y as possible – but larded it with fundamental errors in method, data and conclusions.

For starters, the purported new cancer drug was tested on cancer cells – but not healthy cells. So there's no way to tell whether its effect was cancer-specific, or if it's simply toxic to all cells.

A graph in the paper purports to show that the more lichen drug that was added to test tubes of cancer cells, the more effective it was at killing. But in fact the actual data show no such difference.

Bohannan says it wasn't easy to write a convincing fake. Initially he made the data "too crazy," he says. His Harvard collaborators worried it made the paper look too interesting. "So we rewrote it, making boring rookie mistakes," he says.

The final touch was to make the paper read as though it had been written by someone whose first language is not English. To do that, Bohannan used Google Translate to put it into French, then translated that version back into English.

In the end, the paper's fictitious authors got 157 acceptance letters and 98 rejections – a score of 61 percent. "That's way higher than I expected," Bohannan says. "I was expecting 10 or 15 percent, or worst case, a quarter accepted."

For the privilege of being published, the paper's authors were asked to send along a publishing fee of up to $3,100.

The highest density of acceptances was from journals based in India, where academics are under intense pressure to publish in order to get promotions and bonuses.To learn the location of online journals that accepted or rejected Bohannan's paper, see this interactive global map.

Bohannan says the exercise is a damning indictment of the way peer review works (or doesn't) at many online journals. Peer review is the time-honored system of having outside experts comb through submissions to identify flaws in method, data or conclusions. It's the way scientific journals do quality control.

"Peer review is in a worse state than anyone guessed," he says.

Bohannan says he doesn't mean to suggest that the whole business model of online open-access journals is a failure. "You can't conclude that from my experiment, because I didn't do the right control – submitting a paper to paid-subscription journals," he says.

As he acknowledges, it's not as if peer review is always up to snuff at subscription journals – even the top subscription journals have been embarrassed by lapses in their peer review processes. But he says online publishing makes poor-quality journals easier to set up. And the sheer volume of online publications these days makes it harder to distinguish between legitimate and shady journals.

Jeffrey Beall of the University of Colorado wasn't surprised in the least by the outcome of Bohannan's sting. "He basically found what I've been saying for years," he tells Shots.

A growing number of online open-access journals "are accepting papers just to earn publishing fees, and as a result science is being poisoned by a lot of bad articles," Beall says.

Beall, a research librarian, is a self-appointed watchdog over open-access publishing. He maintains a list of what he calls "predatory publishers" – those who "exploit the open-access model of publishing for their own profit."

He points out that online publishers operate under an incentive that's just the opposite of traditional scientific journals. Print journals have rigid constraints on how many articles they can publish, so they have to screen out all but the best. And they have subscribers to keep happy, so they have to cultivate reputations as curators of high-quality research.

But online journals don't have to worry about subscribers; they make their money by charging contributors – who have a strong incentive to get published. So "the more papers they publish the more money they make," Beall says.

Two big questions arise out of all this: What damage is done by publish-anything journals? And what can be done about it?

The potential damage is both far-reaching and difficult to quantify. Bohannan points out that universities and government agencies, particularly in developing countries, may hire researchers based on resumes packed with sleazy citations. Determining which of those CV entries is high-quality and which aren't is no easy task.

Beall notes that lawyers often use scientific citations in briefs and trials. Government officials draw on published research to set policy. Drug companies have a strong incentive to manipulate research to bolster their claims. And researchers may be led down futile paths on the basis of poor research.

As to what can be done, Beall says poor-quality research can probably only be driven out by naming and shaming.

Bohannan thinks there might be a sort of Consumer Reports to survey the quality of online journals and call out those that fall short. And he thinks maybe such an enterprise might regularly carry out stings like his to keep everyone in the field on their toes.