I was completely unaware of this, but apparently cases of academic misconduct, as evidenced by the retraction of papers from journals and other publication venues, have been on the rise.
According to the article, retractions from journals in the PubMed database have increased by a factor of 60 over ten years, from 3 in 2000 to 180 in 2009. That’s insane!
What’s going on, then? I suspect one or more of the following:
One caveat: this result derives from PubMed, which primarily includes medical and pharmaceutical research, as well as some auxiliary technology and basic science. Does this pattern of misconduct apply in other fields, or is it particular to medicine?
Improved review processes are necessary, but it’s not clear how quickly change will come. Problems with peer review have been acknowledged for more than 20 years, with a report from 1990 showing that only 8% of members of the Scientific Research Society considered it effective as is. Despite this, in most venues, peer review functions the same way it always has.
There may be some movement, however. CHI, for example, includes the alt.chi track in which research is reviewed in a public forum before selection by a jury, which seems to offer a good compromise between open and free criticism, and peer-driven moderation. There’s also a special conference coming up entitled “Evaluating Research and Peer Review – Problems and Possible Solutions” – it was the Call for Papers for this that got me writing this post.
From my perspective, an ideal research review system would at least:
What else should a review system incorporate? How could such a system fail? Why might it not be adopted?
Update 2012-05-09: It’s not clear whether the aforementioned study relied on the same set of journals each year, or whether they used the full PubMed database each year. It’s probable that the PubMed mix has changed over the decade; for example, the NIH’s public access policy requiring publicly funded research be placed into PubMed was trialed in 2005, and made mandatory in 2008.
I spent Saturday at the HCI for Peace workshop representing the Voices from the Rwandan Tribunal project. It was fairly informal, with only 10 participants, which made it easy for everyone to participate in the discussion. Several participants presented projects they’ve worked on, including:
Also in attendance were Juan Pablo Hourcade, an Assistant Professor at the University of Iowa and organizer of the event; Lisa Nathan, an Assistant Professor at the University of British Columbia, co-PI on the Rwandan project, and a former student at UW; Daniela Busse, from Samsung Research; Daisy Yoo, a student and colleague of mine at UW also working on the Rwandan project, and Kelsey Huebner, an undergraduate assisting Juan-Pablo with running the workshop. Neema Moraveji, director of the Calming Technology Lab at Stanford, was not present, but gave a short presentation on his work in ‘calming technology’ via Skype.
As well as individual project presentations, we also discussed the place of HCI in peace-making, peace-keeping, and harmony. A number of points and questions were salient:
In the time available, it was impossible to come to any detailed consensus on these issues, and it was generally agreed that further thought and development would be necessary. Interactions magazine has offered us a spot as the cover article in an issue later this year, and we’re hoping that this will give us an opportunity to address these concerns in more depth.
All up, a fascinating and rewarding way to spend a day. Not to mention an excellent lunch and tasty pizza and conversation at the end of the day!