Science publishing: The trouble with retractions - NATURE

A surge in withdrawn papers is highlighting weaknesses in the system for handling them.

Richard Van Noorden

This week, some 27,000 freshly published research articles will pour into the Web of Science, Thomson Reuters' vast online database of scientific publications. Almost all of these papers will stay there forever, a fixed contribution to the research literature. But 200 or so will eventually be flagged with a note of alteration such as a correction. And a handful — maybe five or six — will one day receive science's ultimate post-publication punishment: retraction, the official declaration that a paper is so flawed that it must be withdrawn from the literature.

It is reassuring that retractions are so rare, for behind at least half of them lies some shocking tale of scientific misconduct — plagiarism, altered images or faked data — and the other half are admissions of embarrassing mistakes. But retraction notices are increasing rapidly. In the early 2000s, only about 30 retraction notices appeared annually. This year, the Web of Science is on track to index more than 400 (see 'Rise of the retractions') — even though the total number of papers published has risen by only 44% over the past decade. 

Perhaps surprisingly, scientists and editors broadly welcome the trend. "I don't think there's any doubt that we're detecting more fraud, and that systems are more responsive to misconduct. It's become more acceptable for journals to step in," says Nicholas Steneck, a research ethicist at the University of Michigan in Ann Arbor. But as retractions become more commonplace, stresses that have always existed in the system are starting to show more vividly.

When the UK-based Committee on Publication Ethics (COPE) surveyed editors' attitudes to retraction two years ago, it found huge inconsistencies in policies and practices between journals, says Elizabeth Wager, a medical writer in Princes Risborough, UK, who is chair of COPE. That survey led to retraction guidelines that COPE published in 2009. But it's still the case, says Wager, that "editors often have to be pushed to retract".

Other frustrations include opaque retraction notices that don't explain why a paper has been withdrawn, a tendency for authors to keep citing retracted papers long after they've been red-flagged (see 'Withdrawn papers live on') and the fact that many scientists hear 'retraction' and immediately think 'misconduct' — a stigma that may keep researchers from coming forward to admit honest errors.

Perfection may be too much to expect from any system that has to deal with human error in all its messiness. As one journal editor told Wager, each retraction is "painfully unique".

But as more retractions hit the headlines, some researchers are calling for ways to improve their handling. Suggested reforms include better systems for linking papers to their retraction notices or revisions, more responsibility on the part of journal editors and, most of all, greater transparency and clarity about mistakes in research.

The reasons behind the rise in retractions are still unclear. "I don't think that there is suddenly a boom in the production of fraudulent or erroneous work," says John Ioannidis, a professor of health policy at Stanford University School of Medicine in California, who has spent much of his career tracking how medical science produces flawed results.

In surveys, around 1–2% of scientists admit to having fabricated, falsified or modified data or results at least once (D. Fanelli PLoS ONE 4, e5738; 2009). But over the past decade, retraction notices for published papers have increased from 0.001% of the total to only about 0.02%. And, Ioannidis says, that subset of papers is "the tip of the iceberg" — too small and fragmentary for any useful conclusions to be drawn about the overall rates of sloppiness or misconduct.

Instead, it is more probable that the growth in retractions has come from an increased awareness of research misconduct, says Steneck. That's thanks in part to the setting up of regulatory bodies such as the US Office of Research Integrity in the Department of Health and Human Services. These ensure greater accountability for the research institutions, which, along with researchers, are responsible for detecting mistakes.

The growth also owes a lot to the emergence of software for easily detecting plagiarism and image manipulation, combined with the greater number of readers that the Internet brings to research papers. In the future, wider use of such software could cause the rate of retraction notices to dip as fast as it spiked, simply because more of the problematic papers will be screened out before they reach publication. On the other hand, editors' newfound comfort with talking about retraction may lead to notices coming at an even greater rate.

"Norms are changing all the time," says Steven Shafer, editor-in-chief of the journal Anesthesia & Analgesia, who has participated in two major misconduct investigations — one of which involved 11 journals and led to the retraction of some 90 papers.

It's none of your damn business!

But willingness to talk about retractions is hardly universal. "There are a lot of publishers and a lot of journal editors who really don't want people to know about what's going on at their publications," says New York City-based writer Ivan Oransky, executive editor at Reuters Health. In August 2010, Oransky co-founded the blog Retraction Watch with Adam Marcus, managing editor at Anesthesiology News. Since its launch, Oransky says, the site has logged 1.1 million page views and has covered more than 200 retractions.

In one memorable post, the reporters describe ringing up one editor, L. Henry Edmunds at the Annals of Thoracic Surgery, to ask about a paper withdrawn from his journal (see go.nature.com/ubv261). "It's none of your damn business!" he told them. Edmunds did not respond to Nature 's request to talk for this article.

The posts on Retraction Watch show how wildly inconsistent retractions practices are from one journal to the next. Notices range from informative and transparent to deeply obscure. A typically unhelpful example of the genre would be: "This article has been withdrawn at the request of the authors in order to eliminate incorrect information." Oransky argues that such obscurity leads readers to assume misconduct, as scientists making an honest retraction would, presumably, try to explain what was at fault.

To Drummond Rennie, deputy editor of the Journal of the American Medical Association, there are two obvious reasons for obscure retraction notices: "fear and work."

The fear factor, says Wager, is because publishers are very frightened of being sued. "They are incredibly twitchy about publishing anything that could be defamatory," she says.

'Work' refers to the phenomenal effort required to sort through authorship disputes, concerns about human or animal subjects, accusations of data fabrication and all the other ways a paper can go wrong. "It takes dozens or hundreds of hours of work to get to the bottom of what's going on and really understand it," says Shafer. Because most journal editors are scientists or physicians working on a voluntary basis, he says, that effort comes out of their research and clinical time.

But the effort has to be made, says Steneck. "If you don't have enough time to do a reasonable job of ensuring the integrity of your journal, do you deserve to be in business as a journal publisher?" he asks. Oransky and Marcus have taken a similar stance. This summer, for example, Retraction Watch criticized the Journal of Neuroscience for a pair of identical retraction notices it published on 8 June: "At the request of the authors, the following manuscript has been retracted."

But the journal's editor-in-chief, neuroscientist John Maunsell of Harvard Medical School in Boston, Massachusetts, argues that such obscurity is often the most responsible course to take. "My feeling is that there are far fewer retractions than there should be," says Maunsell, who adds that he has conducted 79 ethics investigations in more than 3 years at the journal — 1 every 2–3 weeks. But "authors are reluctant to retract papers", he says, "and anything we put up in the way of a barrier or disincentive is a bad thing. If authors are happier posting retractions without extra information, I'd rather see that retraction go through than provide any discouragement."

At the heart of these arguments, says Steneck, lie shifting norms of how responsible journal editors should be for the integrity of the research process. In the past, he says, "they felt that institutions and scientists ought to do it". More and more journal editors today are starting to embrace the gatekeeper role. But even now, Shafer points out, they have only limited authority to challenge institutions that are refusing to cooperate. "I have had institutions, where I felt there was very clear misconduct, come back and tell me there was none," Shafer says. "And I have had a US institution tell me that they would look into allegations of misconduct only if I agreed to keep the results confidential."

The blame game

Discussions on Retraction Watch make it clear that many scientists would like to separate two aspects of retraction that seem to have become tangled together: cleaning up the literature, and signalling misconduct. After all, many retractions are straightforward and honourable. In July, for example, Derek Stein, a physicist at Brown University in Providence, Rhode Island, retracted a paper in Physical Review Letters on DNA in nanofluidic channels when he found that a key part of the analysis had been performed incorrectly. His thoroughness and speed — the retraction came just four months after publication — were singled out for praise on Retraction Watch.

But because almost all of the retractions that hit the headlines are dramatic examples of misconduct, many researchers assume that any retraction indicates that something shady has occurred. And that stigma may dissuade honest scientists from doing the right thing. One American researcher who talked to Nature about his own early-career retraction said he hoped that his decision would be seen as a badge of honour. But, even years later and with his career established, he still did not want Nature to use his name or give any details of the case.

There is no general agreement about how to reduce this stigma. Rennie suggests reserving the retraction mechanism exclusively for misconduct, but that would require the creation of a new term for withdrawals owing to honest mistakes. At the other extreme, Thomas DeCoursey, a biologist at Rush University Medical Center in Chicago, argues for retraction of any paper that publishes results that are not reproducible. "It does not matter whether the error was due to outright fraud, honest mistakes or reasons that simply cannot be determined," he says.

A better vocabulary for talking about retractions is needed, says Steneck — one acknowledging that retractions are just as often due to mistakes as to misconduct. Also useful would be a database for classifying retractions. "The risk for the research community is that if it doesn't take these problems more seriously, then the public — journalists, outsiders — will come in and start to poke at them," he points out.

The only near-term solution comes back to transparency. "If journals told readers why a paper was retracted, it wouldn't matter if one journal retracted papers for misconduct while another retracted for almost anything," says Zen Faulkes, a biologist at the University of Texas–Pan American in Edinburg, Texas.

Oransky agrees. "I think that what we're advocating is part of a much larger phenomenon in public life and on the Web right now," he says. "What scientists should be doing is saying, 'In the course of what we do are errors, and among us are also people that commit misconduct or fraud. Look how small that number is! And here's what we're doing to root that out.'"

Richard Van Noorden is an assistant news editor for Nature in London. For more analysis of retraction statistics, click here.