Mathematicians have been studying the number pi for thousands of
years, so it might seem startling to learn that a gentleman in Athens,
Wisconsin suddenly changed its value.
His revised pi is a bit bigger than the one everyone else uses. And it stops after 12 digits instead of running on forever.
To any trained mathematician, this isn’t even worth a second look.
It’s the work of an amateur who doesn’t understand pi, the ratio of the
circumference of a circle to the diameter.
But his paper, published in a fake science journal that will print
anything for a fee, now shows up in Google Scholar, including footnotes
citing himself, himself, himself and himself again. Oh, and Pythagoras —
once.
Google Scholar is a search engine that looks for scientific articles and theses, the meat and potatoes of scientific literature.
But Google Scholar is not discerning. It also turns up a new paper
from an Egyptian engineer who decided to rewrite Einstein and claims to
have discovered the nature of dark energy at the same time.
Again, that’s an eye-roller for anyone in the physics business, yet
there it is in a search of scholarly journals, muddying up the
intellectual waters.
It wasn’t supposed to happen this way, of course.
The peer review system was designed to ensure that before research is
published, it’s of good quality, whether everyone agrees with its
conclusions or not.
Under the system, a researcher who makes a discovery sends it to a
science journal to publish. The journal sends it to a group of experts
in the field to check it out to see whether the work is well done. If
the peers approve, it is published — often with changes requested by
these experts.
But peer review is under assault, from both the outside and the inside.
Thousands of “predatory” publishers that imitate science journals are
undermining scientists’ ability to distinguish good from bad.
The predators skip peer review. A series of tests by the Citizen,
Science magazine and others found that many predators will print
anything verbatim, allowing a flood of low-quality work to appear in
online journals.
Researchers in Canada often maintain they are skilled enough to
recognize and ignore worms in the scientific apple. But there’s a harder
question: if simply godawful papers are getting published, what about
the whole blurry spectrum ranging from substandard through so-so to
slightly plagiarized?
It’s not just an academic question. Our daily use of technology, and
even the medical treatments we receive, depend on the ability of
researchers to trade information honestly.
Many academics see the rise of a two-tier science system, a stronger
one found in developed countries, and researchers with fewer
resources in the developing world who depend on predators. A Turkish
newspaper recently wrote that whoever pays the predators “climbs the
career steps two by two.”
There are also small open-access journals struggling to set
themselves up with insufficient resources. One in Ottawa claims to
publish 23 international journals, using freelancers, from a single room
in Billings Bridge. Another operates from a house in Burnaby, B.C.
Staff who work there won’t reveal their last names.
Scam scientific conferences are also a growing assault on the peer
review process. These events allow anyone to register, for a fee, to
present a paper on anything, good or bad, on topic or off.
The speaker can then go to the dean of his or her faculty and say,
“Look, I’m the keynote speaker at a conference in Paris!” It seems like a
career-boosting move, especially if the name of the conference mimics
that of a real scientific gathering.
It’s all about blurring.
WASET, the World Association of Science, Engineering and Technology,
organizes conferences somewhere in the world every few days, but they’re
low-quality affairs at which anyone can register a paper on anything.
To dress it up, WASET offers conferences with names the same or
similar to real conferences organized by real scientific groups.
Recently, WASET put on an International Conference on Educational Data
Mining. The real version belongs to the International Educational Data
Mining Society.
In October alone, the Turkey-based WASET has set up conferences in
Bali, Brussels, Osaka, London, Paris, Dubai, Barcelona and Istanbul.
Each will cover dozens of fields — aviation, agriculture, business
management, linguistics, mathematics, pedagogy, biology, law, medicine,
computer engineering, nanoscience, history, civil engineering, geology,
chemistry, ecology and on and on.
Anyone who presents a paper pays 500 euros, roughly $700, (and 100 euros more if they want the paper published.)
It is tough to shut down groups like WASET. It is not against the law
to stage a conference with the same name as another, and even if
authorities were to go after such operators, they are hard to identify
and could easily pop up under a new name a week later. WASET has
scheduled 103 conferences for next year, mostly in Western Europe and
two in Canada. In most cases, universities pay for the travel.
Canadian academics like to say that we don’t get fooled by scams. But at
the University of King’s College in Halifax, science historian Gordon
McOuat noted that his university has had to cancel travel plans of
faculty wanting to attend scam conferences.
Jeffrey Beall of the University of Colorado found a new publisher of
107 online journals — sprintjournals.com — that advertises an
affiliation with the established Elsevier publishing group. There’s no
actual connection, but anyone reading Sprint’s website will see the
familiar Elsevier logo, and may trust it.
Real journals are “hijacked” by impostors using the same name. So an
article published in Afinidad can be either in a reputable journal from
Spain or an impostor using Afinidad’s name to trick authors. This is a
widespread practice. An Indian publisher has a website mimicking the
legitimate BioMed Central.
The fake journal Experimental & Clinical Cardiology used to be
real until it was sold and new owners took a more profitable path. It
blurs its identity by using the same name as before and by claiming to
be the official journal of a very real and legitimate cardiology
society, although there is no connection.
Legitimate peer review is far from dead. But it has a nasty cough that isn’t clearing up.
The papers on pi and dark energy are just two of many that show up in academic index services.
“For example, Google Scholar does not screen for quality, and it
indexes many articles that contain pseudo-science in them,” Beall says.
“(It) is the world’s most popular index for scholarly content. This
index and many other abstracting and indexing services do not
sufficiently screen for quality and allow much scientific junk to be
included in their databases. This affects the cumulative nature of
science, where new research builds on the research already recorded in
the academic record.”
And the predators have other ways to gain acceptance.
“Some scholarly publishing organizations do not screen applicants for
membership. Thus some predatory publishers apply and are granted
membership. Then the predatory publishers use these memberships to argue
that they are legitimate publishers.”
Even the top ranks of peer review have their problems, though.
Two U.S. psychologists ran a test of peer review back in 1982, taking
12 papers that had been published in high-ranking psychology journals
and re-submitting them to the same journals with changed titles and
different authors’ names.
All the originals came from famous institutions. The re-submitted
ones carried names of fictional authors and institutions, some on
the hippy-dippy side: the “Tri-Valley Center for Human Potential,” for
instance.
The same body of work should be accepted again, the two
researchers felt. It wasn’t. Only three of 38 reviewers and editors
spotted the duplicates. That allowed nine papers to continue, and eight
were turned down for allegedly poor quality — evidence of bias, the
authors concluded, against academic institutions with lower pedigree.
(Giving preference to well-known academics is known as the Matthew
Effect, from a passage in the Gospel of Matthew: “For to everyone who
has, more shall be given…”) The journal Science flagged it as a problem
as far back as 1968.
Yes, but people are smarter now, right?
Not so fast. In 2006, the editor of the British Medical Journal
(BMJ), Richard Smith, listed problems with peer review. It’s
inconsistent, he found: two reviewers of the same paper can come to
“laughably” opposite conclusions. Sometimes it’s dishonest (as when a
reviewer rejected a paper, but stole chunks of it for his own work.) It
rarely catches fraud. And it’s tilted in favour of male researchers.
Smith, who was also chief executive of the BMJ Publishing
Group, calls peer review “little better than tossing a coin” and “a
flawed process, full of easily identified defects with little evidence
that it works.”
And he cites a colleague with similar misgiving: “That is why Robbie
Fox, the great 20th century editor of the Lancet, who was no admirer of
peer review, wondered whether anybody would notice if he were to swap
the piles marked ‘publish’ and ‘reject.'”
Its nature, he writes, is impossible to define precisely. “Peer
review is thus like poetry, love, or justice.” (A side note: Smith
rented a 15th-century palazzo in Venice to write this as part of a
longer analysis, and he clearly had the arts on his mind, comparing a
medical study at one stage with an altarpiece by Tintoretto.)
Still, Smith notes, the scientific establishment believes in peer
review, and concludes wryly: “How odd that science should be rooted in
belief.”
This summer the British Medical Journal published a study of how well peer review worked in 93 recent medical trials.
It explained: “Despite the widespread use of peer review little is
known about its impact on the quality of reporting of published research
articles.”
It concluded that “peer reviewers often fail to detect important
deficiencies in the reporting of the methods and results
of randomized trials,” and they “requested relatively few changes for
reporting of trial methods and results.” Most of their suggestions were
helpful but a few were not, it added.
Sometimes, even with the names removed, reviewers may recognize an
author’s research because everyone works in the same field. Christine
Wenneras and Agnes Wold, two Swedes analyzing peer review in Nature in
1997, wrote of the ”friendship bonus” and “nepotism” that can occur.
As a widely quoted 2006 opinion piece in Nature by Charles G.
Jennings, one of the journal’s former editors, noted: “Scientists
understand that peer review per se provides only a minimal assurance of
quality, and that the public conception of peer review as a stamp of
authentication is far from the truth.” Jennings went on to argue for
harder, more quantifiable factors to make peer review more dependable —
work currently under way by an international group called EQUATOR which
promotes strict reporting guidelines.
Medical professor Roger Pierson of the University of Saskatchewan points to the flip side: rivalry.
“Human nature being what it is, professional jealousy and egos flare
from time to time just because people don’t like each other — they’ll
trash each others’ manuscripts to be spiteful,” he wrote in an email.
“There are also constant cases of people in races to claim ‘First!!!’
for whatever that’s worth,” and giving a negative or delayed review to
one’s rival “can give them an advantage,” he wrote. “The science
community and the university system have no real way to respond or
ensure that their members are playing nicely with one another. C’est la
vie.”
These days, some scientists skip the whole traditional publishing
process, at least for some of their work. The Internet beckons, and they
go straight to their audience, cutting out the middleman.
This is what University of Ottawa biologist Jules Blais calls “the
blogification of science.” It doesn’t replace traditional journal
publishing, but “this is something that we have been seeing with social
media. The volume has gone way up and the quality is coming down. We
have to be very careful in how we preserve our highly regarded
peer-reviewed publications because we need them desperately.”
Another way to bypass peer review is to post work directly online at
arXiv (pronounced “archive,”) hosted by Cornell University. It takes
papers in mathematics and some sciences, including physics and
astronomy. The system is called “preprint,” implying that papers can go
online at arXiv while awaiting peer review somewhere else. But the
second stage isn’t mandatory, and there are now more than 8,000 papers a
month posted on arXiv.
Still, they can’t ignore the traditional journals entirely. Careers
are built there. Nature estimates academics worldwide publish more than
one million papers a year.
“Everything we do is really judged on publications, and if we want
grant funding (to keep a lab running), people look at your CV,” says
Joyce Wilson, a virus researcher and relatively new associate professor
at the University of Saskatchewan. “And if you have published well in
the past they assume that you will publish well in the future, and they
will give money.”
All of this worries Saskatchewan’s Pierson. “Peer review doesn’t
catch many things, even patent fraud,” he says. “There is a growing
literature on this one. Fraud and deceit in the halls of science have
been around forever and now that careerism seems to have become more
important than the search for truth that many, if not most, of us
actually entered the biz to pursue…. well, things have progressed.
“Even the biggest, most prestigious journals are not immune. There have
been some particularly egregious cases in the past decade. So, cutting
the garbage? Perhaps not as much as we would like to believe.”
Still, he concludes, that while peer review is far from perfect,
“right now it’s the best we’ve got. I think that the system needs an
overhaul and perhaps this issue (bogus science) is a good stimulus.”
