Diederik Stapel has been a psychology professor at major universities for the last ten years. He published well over 100 research papers in prestigious journals such as Science. Some of his research papers have been highly cited. He trained nearly 20 Ph.D. students.
It was recently fired when it was finally determined that he was making up all of his research data, including the data that he was providing to students. He was making up research assistants and experiments. He wasn’t even particularly careful as the data had significant statistical anomalies (such as identical averages for different data sets).
Managers, colleagues, journals, collaborators and competitors failed to openly report him. It took outsiders (students) to report him. The best journals, and correspondingly, the best scientists were repeatedly fooled by Stapel. Judging by his numerous citations, people built on his work…
People who want to believe that “peer reviewed work” means “correct work” will object that this is just one case. But what about the recently dismissed Harvard professor Marc Hauser? We find exactly the same story. Marc Hauser published over 200 papers in the best journals, making up data as he went. Again colleagues, journals and collaborators failed to openly challenge him: it took naive students, that is, outsiders, to report the fraud.
The real scientists, the peers of the researchers, don’t report fraud. Questioning someone’s results is a dangerous adventure.
Some point out to me that this does not apply to fields such as Computer Science. Really? Have you ever tried to reproduce the experimental results from popular papers? Quite often, it is very difficult or even impossible. It does not help that Computer Science researchers almost never post their software or data. (Almost all my software is already online.)
But what is critical is that traditional peer review does not protect against fraud. It is merely a check that the work appears superficially correct and interesting. A reviewer who would go out of his way to check whether a paper reports truthful results should not expect accolades. That is not how the game is played.
Further reading: How reliable is science?
No comments:
Post a Comment